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The  NATO  Science  and  Technology  Organization 


Science  &  Technology  (S&T)  in  the  NATO  context  is  defined  as  the  selective  and  rigorous  generation  and  application  of 
state-of-the-art,  validated  knowledge  for  defence  and  security  purposes.  S&T  activities  embrace  scientific  research, 
technology  development,  transition,  application  and  field-testing,  experimentation  and  a  range  of  related  scientific 
activities  that  include  systems  engineering,  operational  research  and  analysis,  synthesis,  integration  and  validation  of 
knowledge  derived  through  the  scientific  method. 

In  NATO,  S&T  is  addressed  using  different  business  models,  namely  a  collaborative  business  model  where  NATO 
provides  a  forum  where  NATO  Nations  and  partner  Nations  elect  to  use  their  national  resources  to  define,  conduct  and 
promote  cooperative  research  and  information  exchange,  and  secondly  an  in-house  delivery  business  model  where  S&T 
activities  are  conducted  in  a  NATO  dedicated  executive  body,  having  its  own  personnel,  capabilities  and  infrastructure. 

The  mission  of  the  NATO  Science  &  Technology  Organization  (STO)  is  to  help  position  the  Nations’  and  NATO’s  S&T 
investments  as  a  strategic  enabler  of  the  knowledge  and  technology  advantage  for  the  defence  and  security  posture  of 
NATO  Nations  and  partner  Nations,  by  conducting  and  promoting  S&T  activities  that  augment  and  leverage  the 
capabilities  and  programmes  of  the  Alliance,  of  the  NATO  Nations  and  the  partner  Nations,  in  support  of  NATO’s 
objectives,  and  contributing  to  NATO’s  ability  to  enable  and  influence  security  and  defence  related  capability 
development  and  threat  mitigation  in  NATO  Nations  and  partner  Nations,  in  accordance  with  NATO  policies. 

The  total  spectrum  of  this  collaborative  effort  is  addressed  by  six  Technical  Panels  who  manage  a  wide  range  of 
scientific  research  activities,  a  Group  specialising  in  modelling  and  simulation,  plus  a  Committee  dedicated  to 
supporting  the  information  management  needs  of  the  organization. 


AVT  Applied  Vehicle  Technology  Panel 

HFM  Human  Factors  and  Medicine  Panel 
1ST  Information  Systems  Technology  Panel 

NMSG  NATO  Modelling  and  Simulation  Group 

SAS  System  Analysis  and  Studies  Panel 
SCI  Systems  Concepts  and  Integration  Panel 

SET  Sensors  and  Electronics  Technology  Panel 


These  Panels  and  Group  are  the  power-house  of  the  collaborative  model  and  are  made  up  of  national  representatives  as 
well  as  recognised  world-class  scientists,  engineers  and  information  specialists.  In  addition  to  providing  critical 
technical  oversight,  they  also  provide  a  communication  link  to  military  users  and  other  NATO  bodies. 


The  scientific  and  technological  work  is  carried  out  by  Technical  Teams,  created  under  one  or  more  of  these  eight 
bodies,  for  specific  research  activities  which  have  a  defined  duration.  These  research  activities  can  take  a  variety  of 
forms,  including  Task  Groups,  Workshops,  Symposia,  Specialists’  Meetings,  Lecture  Series  and  Technical  Courses. 


The  content  of  this  publication  has  been  reproduced  directly  from  material  supplied  by  STO  or  the  authors. 


Published  September  2014 

Copyright  ©  STO/NATO  2014 
All  Rights  Reserved 

ISBN  978-92-837-2005-8 


Single  copies  of  this  publication  or  of  a  part  of  it  may  be  made  for  individual  use  only  by  those  organisations  or 
individuals  in  NATO  Nations  defined  by  the  limitation  notice  printed  on  the  front  cover.  The  approval  of  the  STO 
Information  Management  Systems  Branch  is  required  for  more  than  one  copy  to  be  made  or  an  extract  included  in 
another  publication.  Requests  to  do  so  should  be  sent  to  the  address  on  the  back  cover. 
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Operations  Research  /  Operations  Analysis 
Orientation  Course  Curriculum  for  NATO  Nations 

(STO-TR-SAS-098) 

Executive  Summary 

SAS-098  was  tasked  to  design  a  set  of  instructional  modules  to  demonstrate  the  value  of  various  Operations 
Research  and  Operational  Analysis  (OR/OA)  analytical  tools  in  decision-making  to  the  non-analyst. 
The  annex  to  this  report  provides  these  modules  categorized  by  application  area.  In  addition  to  inclusion  in 
this  report’s  annex,  instructional  modules  are  posted  on  the  ACT  website  (https://transnet.act.nato.int/WISE/ 
NATOAnalys)  for  NATO  instructors  and  analysts  to  review  and  download  for  use.  Specific  delivery  venues 
may  be  in  NATO  senior  executive  exercises,  an  STO  Lecture  Series,  inclusion  in  existing  NATO  schools 
and  courses,  and  by  NATO  Nations  within  their  national  course  offerings.  Post-study,  it  is  envisioned  that 
instructors  may  wish  to  use  the  templates  offered  here  to  develop  case  studies  on  future  operations  so  that  the 
bank  of  instruction  will  grow,  be  timely,  and  relevant. 

STUDY’S  OBJECTIVE 

Commanders  and  their  staffs  must  have  an  appreciation  for  the  potential  of  OR/OA  analytical  tools  to  fully 
leverage  their  potential.  This  study  designs  a  set  of  instructional  modules  to  demonstrate  to  non-analysts  how 
OR/OA  is  useful  in  their  decision-making  and  operational  assessment  challenges. 


SCOPE,  ORGANIZATION,  AND  FORMAT 

We  select  case  studies  as  the  basic  format  for  the  instructional  modules.  Each  case  study  has  a  real-world 
example  to  demonstrate  the  relevancy  of  the  operational  analysis  techniques  to  decision-makers.  The  team 
identified  six  application  areas  to  consider  for  case  study  generation.  These  application  areas  are  strategic 
planning,  support  of  military  operations,  transforming  armed  forces,  logistics  and  acquisition,  scheduling, 
and  other.  The  “strategic  planning”  application  area  addresses  case  studies  of  national  or  strategic 
importance.  “Support  of  military  operations”  application  area  focuses  on  case  studies  dealing  with  decisions 
or  operations  at  the  operational  level  usually  while  the  operations  are  on-going  or  being  planned. 
The  application  area  title  “Transforming  Armed  Forces”  deals  with  case  studies  related  to  re-shaping  force 
structures,  processes,  or  bases.  “Logistics  and  acquisition”  application  area  contains  decision  related  cases 
dealing  with  supply,  support,  or  the  procurement  of  forces.  “Scheduling”  deals  with  cases  dealing  with  the 
efficient  employment  of  forces  or  logistics  that  have  an  operational  impact.  The  “other”  application  area 
allows  for  the  introduction  of  basic  operational  analysis  tools  and  for  case  studies  not  falling  neatly  into  any 
other  application  category  respectively. 

Individual  case  study  modules  have  three  components;  an  instructional  module  summary,  the  actual 
presentation  material,  and  any  background  reading  material  for  future  instructors  and  executives. 
Each  instructional  module  summary  and  presentation  has  a  standard  format.  The  instructional  module 
summary  includes  the  title  of  the  case  study,  the  developer’s  name  and  organization,  presentation  length  in 
time,  learning  objectives,  a  brief  outline,  any  additional  materials,  and  delivery  methods.  In  general, 
the  presentation  has  two  parts:  a  ten  to  twenty  minute  summary  for  a  general  officer,  flag  officer, 
or  executive  decision-makers;  and  a  second  part  for  more  junior  staff  officers  or  analysts  wishing  more 
detail.  Each  presentation  follows  a  template,  with  the  understanding  that  future  presenters  may  modify  it  to 
suit  their  audience. 
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Programme  d’initiation  a  la  recherche  /  l’analyse 
operationnelle  pour  les  pays  de  l’OT AN 
(STO-TR-SAS-098) 


Synthese 

Le  SAS-098  avait  pour  mission  de  concevoir  un  ensemble  de  modules  destruction  demontrant  la  valeur  de 
differents  outils  analytiques  de  recherche  et  d’ analyse  operationnelles  (OR/OA)  dans  la  prise  de  decision  des 
non-analystes.  L’annexe  au  present  rapport  presente  ces  modules  classes  par  domaine  d’ application. 
Par  ailleurs,  les  modules  destruction  sont  places  sur  le  site  Internet  (https://transnet.act.nato.int/WISE/ 
NATOAnalys)  pour  que  les  instructeurs  et  analystes  de  l’OTAN  les  lisent,  les  telechargent  et  les  utilisent. 
Ces  modules  sont  susceptibles  d’etre  presentes  lors  d’exercices  majeurs  de  l’OTAN  ou  dans  une  serie  de 
conferences  de  la  STO ;  ils  peuvent  etre  inclus  dans  les  cours  et  ecoles  de  l’OTAN  existants  et  les  cours 
nationaux  delivres  par  les  pays  de  l’OTAN.  Apres  l’etude,  les  instructeurs  auront  la  possibility  d’utiliser  les 
modeles  proposes  ici  pour  elaborer  des  etudes  de  cas  sur  des  operations  futures,  afin  que  la  banque 
d’ instructions  s’etoffe,  reste  d’actualite  et  demeure  pertinente. 


OBJECTIF  DE  L’ETUDE 

Les  commandants  et  leurs  etats-majors  doivent  mesurer  le  potentiel  des  outils  analytiques  d’ OR/OA  pour  les 
exploiter  pleinement.  La  presente  etude  etablit  un  ensemble  de  modules  d’ instruction  dans  le  but  de 
demontrer  aux  non-analystes  en  quoi  1’  OR/OA  est  utile  a  leur  prise  de  decision  et  aux  difficultes 
d’ evaluation  operationnelle. 


PORTEE,  STRUCTURE  ET  FORMAT 

Le  format  de  base  des  modules  destruction  est  l’etude  de  cas.  Chaque  etude  de  cas  s’accompagne  d’un 
exemple  du  monde  reel  afin  de  prouver  aux  decideurs  la  pertinence  des  techniques  d’ analyse  operationnelle. 
L’equipe  a  identifie  six  domaines  d’application  a  etudier  pour  la  production  des  etudes  de  cas.  Ces  domaines 
d’ application  sont  la  planification  strategique,  le  soutien  des  operations  militaires,  la  transformation  des 
forces  armees,  la  logistique  et  P  acquisition,  la  programmation  et  une  categorie  appelee  « autres ». 
Le  domaine  d’application  «  planification  strategique  »  conceme  les  etudes  de  cas  d’ importance  nationale  ou 
strategique.  Le  domaine  d’application  «  soutien  des  operations  militaires  »  se  concentre  sur  les  etudes  de  cas 
ayant  trait  a  des  decisions  ou  operations  au  niveau  operationnel,  d’ ordinaire  pendant  le  deroulement  ou  la 
planification  des  operations.  Le  domaine  d’application  « transformation  des  forces  armees  »  s’interesse  aux 
etudes  de  cas  liees  a  la  modification  des  structures,  processus  ou  bases  des  forces  armees.  Le  domaine 
d’application  « logistique  et  acquisition  »  contient  les  cas  relatifs  aux  decisions  de  foumiture,  de  soutien  ou 
d’ appro visionnement  des  forces.  Le  domaine  «  programmation  »  regroupe  les  etudes  de  cas  qui  traitent  de 
1 ’utilisation  efficace  des  forces  ou  de  la  logistique  ayant  un  impact  operationnel.  Le  domaine  d’application 
intitule  «  autres  »  permet  de  presenter  des  outils  d’ analyse  operationnelle  de  base  et  des  etudes  de  cas  qui 
n’entrent  pas  exactement  dans  une  autre  categorie  d’application. 

Les  modules  des  etudes  de  cas  se  composent  de  trois  parties  :  un  resume  du  module  d’ instruction, 
la  presentation  proprement  dite  et  des  references  bibliographiques  pour  les  fiiturs  instructeurs  et  cadres. 
Chaque  resume  et  presentation  des  modules  d’ instruction  a  un  format  standard.  Le  resume  du  module 
d’instruction  inclut  le  titre  de  l’etude  de  cas,  le  nom  et  l’organisme  du  redacteur,  la  duree  de  la  presentation, 
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les  objectifs  d’apprentissage,  une  breve  description,  des  documents  complementaires  et  les  methodes 
d’expose.  En  general,  la  presentation  tient  en  deux  parties  :  un  resume  en  dix  ou  vingt  minutes  destine  a  un 
officier  general,  commandant  ou  cadre  dirigeant  et  une  seconde  partie  pour  les  officiers  de  rang  inferieur  ou 
les  analystes  souhaitant  plus  de  detail.  Chaque  presentation  repose  sur  un  modele,  etant  entendu  que  les 
futurs  presentateurs  peuvent  le  modifier  pour  s’ adapter  a  leur  public. 


STO-TR-SAS-098 


ES-3 


NATO 

"W 

i _ 

OTAN 

organization 


ES  -  4 


STO-TR-SAS-098 


/^NATO 
Wr  Iotan" 


organization 


Chapter  1  -  INTRODUCTION  AND  REPORT  OUTLINE 

Professor  of  Practice,  Jeff  Kline,  Captain,  USN  (ret) 

Naval  Postgraduate  School 
1411  Cunningham  Road,  Room  236 
Monterey,  CA  93943 
UNITED  STATES 


1.1  STUDY  OBJECTIVES 

The  charter  of  SAS-098  Task  Group  043  was  to  design  a  set  of  instructional  modules  to  demonstrate  to  the 
non-analyst  the  value  provided  by  various  Operations  Research  and  Operational  Analysis  (OR/OA) 
analytical  tools  in  decision  making.  Annex  A  provides  these  modules  categorized  by  application  area. 
The  modules  are  also  available  through  the  NATO  Analysis  Community  of  Interest  website  at  https://trans 
net.act.nato.int/WISE/NATOAnalys.  These  modules  are  developed  for  educators,  trainers,  and  practicing 
analysts  for  presentation  to  individual  decision  makers  and  their  staffs,  or  integration  into  existing  NATO 
courses  with  the  goal  of  inspiring  greater  use  of  operational  analysis  NATO-wide. 

The  modules’  primary  audience  are  decision-makers  at  the  flag  and  executive  civilian  level. 

This  report  serves  as  one  delivery  venue  of  the  instruction  modules  and  this  chapter  provides  an  outline  of 
the  report.  Future  case  study  development  may  be  aided  by  the  report’s  display  of  general  format  and 
templates  agreed  upon  by  Member  Nations. 


1.2  BACKGROUND 

As  a  professional  group,  the  military  personnel  of  NATO  Nations  receive  high-level,  quality  education  from 
a  wide  range  of  academic  disciplines  throughout  their  career.  The  need  for  high-quality  analysis  to  support 
decision-making  in  military  acquisition,  in  force  development,  in  force  employment  and  in  management  of 
the  national  military  and  NATO  organizations  compels  continued  emphasis  on  scientific  rigor.  OR/OA 
provides  evidence-based  support  to  decision-making  with  an  emphasis  on  this  scientific  rigor.  Deepening  the 
understanding  of  the  contributions  OR/OA  analytical  tools  provide  will  help  Commanders  and  their  staffs  to 
more  fully  leverage  their  potential.  This  study  designs  a  set  of  instructional  modules  to  demonstrate  to  non¬ 
analysts  how  OR/OA  is  useful  in  their  decision-making  and  operational  assessment  challenges. 


1.3  STUDY  TEAM  COMPOSITION  AND  ACTIVITIES 

The  SAS-098  study  team  includes  participants  from  eight  Nations  listed  in  the  Membership  List.  The  nature 
of  the  study  allowed  for  both  physical  and  virtual  (web)  meetings  to  scope  the  material  to  the  intended 
audience,  develop  application  areas  of  interest,  agree  upon  case  study  composition  and  format,  and  begin  to 
develop  the  material.  In  total,  seven  physical  meetings  and  thirteen  web-based  meeting  were  held  as  listed  in 
Table  1-1.  In  addition  to  these  meetings,  a  working  web  site  was  established  at  the  Naval  Postgraduate 
School  to  collect  draft  module  submissions  and  allow  team  members  to  review  submitted  work 
independently. 
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Table  1-1:  SAS-098  RTG-043  Meeting  Activity  from  16  September  2011  to  27  September  2013. 


Meeting  Date 

Type 

Location 

Comments 

16  September  201 1 

Physical  Meeting 

CSO,  Neuilly-sur-Seine, 
France 

Exploratory  Meeting 

18  January  2012 

Virtual  Meeting 

Web 

Web  Working  site 
established 

21  February  2012 

Physical  Meeting 

Brno,  CHZ 

22  March  2012 

Virtual  Meeting 

Web 

2-3  May  2012 

Physical  Meeting 

Ottobrunn,  DEU 

Application  areas  and 
Instructional  templates 
finalized 

15  June  2012 

Virtual  Meeting 

Web 

11  July  2012 

Virtual  Meeting 

Web 

22  August  2012 

Virtual  Meeting 

Web 

19  September  2012 

Virtual  Meeting 

Web 

16-17  October  2012 

Physical  Meeting 

Monterey,  USA 

Demonstrated  several 
case  studies  to  SAS 

Panel  members  and  flag 
officers,  case  study  list 
finalized 

27  November  2012 

Virtual  Meeting 

Web 

23  January  2013 

Virtual  Meeting 

Web 

20  February  2013 

Virtual  Meeting 

Web 

20  March  2013 

Virtual  Meeting 

Web 

17  April  2013 

Virtual  Meeting 

Web 

10  May  2013 

Virtual  and  Physical 
Meeting 

Web  and  Monterey 

Proposed  several  case 
studies  as  final  versions 

10-11  June  2013 

Physical  Meeting 

Munich,  DEU 

Reviewed  final  report, 
finalized  several  more 
case  studies,  and 
structured  ACT  website 
for  receipt  of  case 
studies 

7  August  2013 

Virtual  Meeting 

Web 

26-27  September  2013 

Physical  Meeting 

Ankara,  Turkey 

Case  study  delivery 
(beta),  review  final 
briefing,  review  final 
report 
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1.4  STUDY  OUTLINE 

The  chapters  of  this  report  provide  information  on  the  development  of  the  case  study  modules  and  templates 
to  use  for  future  development.  This  chapter  introduces  the  study’s  objectives,  discusses  the  background, 
shows  the  study  team  activity,  and  provides  the  remaining  outline.  The  following  table  (Table  1-2)  provides 
the  remaining  chapters’  title  and  a  brief  description  of  their  content. 


Table  1-2:  Report  Organization  by  Chapter. 


Chapter 

Title 

Contents 

1 

Introduction  and  Report  Outline 

Overview 

2 

Scope  and  Case  Studies  Focus 

Past  Studies,  intended  audience, 
case  study  focus,  application 
areas  and  description  of  topics 
developed,  the  OA  tools  matrix 

3 

Instructional  Module  Templates 

Composition  of  the  three-part 
case  study:  module  composition 
introduction;  the  instructional 
module  summary;  and  the 
presentation  template 

4 

Delivery  and  Archiving  Recommendations 

Archived  locations  of  case 
studies  instruction  material  and 
alternative  delivery  venues  and 
summary 

Annex 

Application  Areas  with  Supporting  Curriculum 

Individual  Case  Study  Modules 
Developed 
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Chapter  2  -  SCOPE  AND  CASE  STUDIES  FOCUS 

Professor  of  Practice,  Jeff  Kline,  Captain,  USN  (ret) 

Naval  Postgraduate  School 
1411  Cunningham  Road,  Room  236 
Monterey,  CA  93943 
UNITED  STATES 

2.1  REVIEW  OF  OPERATIONAL  ANALYSIS  STUDIES 

Several  SAS  studies  have  addressed  or  are  addressing  effective  implementation  of  operational  analysis  in 
NATO  operations.  As  summarized  below,  most  of  these  studies  focus  on  aiding  analysts  in  their  efforts  to 
integrate  with  NATO  planning  and  execution  processes. 

A  review  of  the  SAS  studies  summaries  in  Table  2-1  show  these  studies  and  initiatives  are  valuable 
resources  for  a  NATO  operational  analyst.  They  do  not,  however,  address  the  value  of  operational  analysis 
from  a  senior  decision  maker’s  viewpoint  (the  SAS-098  study  group  tasking). 


Table  2-1 :  Past  SAS  Studies  and  Efforts  Addressing  Employment  of  Operational  Analysis. 


Project 

Date 

SAS 
Study  # 

Summary 

Operational  Analysis 
Course  for  Deploying 
Analysts 

In 

Progress 

101 

This  follow-on  effort  to  SAS-89  is  developing  and 
delivering  a  short  course  to  deploying  NATO  analyst. 

Operational  Analysis 
Support  to  NATO 
Operations 

Aug  2012 

089 

SAS-089  “Operational  Analysis  (OA)  Support  to  NATO 
Operations”  elaborates  a  recommendation  of  an 
organizational  structure,  common  procedures  and  training 
that  would  improve  effectiveness,  interoperability, 
and  sustainability  of  OA  support  to  operations  in  both 

NATO  and  national  static  and  deployed  HQs.  The  report 
describes  an  employment  concept  for  OA  support  to  NATO 
operations.  It  outlines  the  roles  and  responsibilities  of  OA, 
describes  where  operational  analysts  should  be  positioned 
in  deployed  HQs,  and  makes  proposals  with  regard 
to  the  function  and  role  of  a  NATO  reach  back  capability, 
as  well  as  qualification,  selection  and  training  of  OA 
personnel.  It  identifies  gaps  between  the  current  and  desired 
situation  and  provides  recommendations  to  fill  these  gaps 
and  for  the  implementation  of  the  proposed  concept. 

NATO  Guide  for 
Judgment-Based 
Operational  Analysis  in 
Defence  Decision- 
Making 

Jun  2012 

087 

A  Guide  consisting  of  three  volumes:  an  analyst-oriented 
document  (the  “Code  of  Best  Practice  for  ‘Soft’ 

Operational  Analysis”),  a  client-oriented  document,  and  a 
brief  summarizing  brochure  for  high-level,  ‘executive’ 
decision-makers. 

STO-TR-SAS-098 


2-1 


SCOPE  AND 

CASE  STUDIES  FOCUS 


organization 


Project 

Date 

SAS 

Study# 

Summary 

NATO  Guide  for 
Judgment-Based 
Operational  Analysis  in 
Defence  Decision- 
Making  (cont’d) 

Jun  2012 

087 

The  purpose  of  the  Guide  is  to  create  an  understanding  of 
what  judgment-based  OA  is;  to  clarify  what  judgment- 
based  OA  can  do  to  help  address  problematic  situations; 
what  people  can  expect  from  it  in  that  respect;  and  to 
provide  guidance  on  how  a  judgment-based  OA  study 
should  be  carried  out  to  maximize  the  validity,  credibility 
and  acceptance  of  the  study  and  its  outcomes. 

NATO  Operations 
Assessment:  A  Case 

Study  Based  on  Planning 
for  Transition  in 
Afghanistan 

Apr  2012 

091 

A  case  study  on  transition  operations  in  Afghanistan. 

The  Human 

Environment  Analysis 
Reasoning  Tool 
(HEART)  - 
Incorporating  Human 
and  Social  Sciences  into 
NATO  Operational 
Planning  and  Analysis 

Nov  2011 

074 

The  paper  provides  an  outline  description  and  user  guide 
for  the  Human  Environment  Analysis  Reasoning  Tool 
(HEART)  which  helps  military  staff  and  analysts 
understand  the  human  and  social  environment,  develop 
effective  courses  of  action,  and  make  use  of  appropriate 
analysis  methods.  HEART  can  be  used  to  support  early 
phases  of  operational  planning,  as  a  training  resource  and  to 
improve  the  representation  of  the  human  and  social 
environment  in  exercises  and  experiments. 

Long  Range  Forecasting 
of  the  Security 
Environment 

Apr  2011 

088 

The  theme  for  the  SAS-088  Specialists  Meeting  was  “Long 
Range  Forecasting  of  the  Security  Environment”.  The 
meeting  aimed  to  establish  a  forum  to  review  national 
and  international  perspectives,  methods  and  supporting 
analytical  techniques  for  long  range  foresight.  To  achieve 
these  goals  the  meeting  was  organized  into  three  sessions 
each  structured  with  a  Keynote  presentation,  a  number  of 
papers,  a  discussion  period,  and  finally  a  syndicate  session 
addressing  a  specific  topic  raised  in  the  session.  These  three 
sessions  covered: 

1)  Methodology  -  Various  approaches,  methods  and  tools 
for  performing  medium-to-long  term  forecasts  of  the 
future  security  environment  intended  to  improve 
defence  policy  and  planning  within  NATO,  and  the 
assumptions,  limitations  and  bias  that  may  challenge 
the  conduct  of  effective  security  analysis. 

2)  Process  -  How  uncertainty  is  accounted  for  within 
various  foresight  approaches  and  its  effect  on 
assessment,  validation  and  measures  of  robustness,  and 
ways  to  respond  to  uncertainty  in  the  development  of 
national  defence  and  security  requirements. 

3)  Integration  with  Policy  -  How  Allied  Nations  conduct 
and  implement  foresight  generation  into  policy 
formulation  and  decision-making. 
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Project 

Date 

SAS 

Study# 

Summary 

Analytical  Support  to 
Defence  Transformation 

Apr  2010 

081 

The  SAS-081  Symposium  on  “Analytical  Support  to 

Defence  Transformation”  was  held  in  Sofia,  Bulgaria, 

26-28  April  2010.  It  aimed  to  provide  a  forum  for  the 
analytical  community  to  address  and  respond  to  NATO’s 
military  transformation  needs.  It  brought  together  90 
participants  from  19  NATO  and  Partner  Nations,  and 
featured  33  presentations,  structured  around  five  main 
themes:  general  analytical  issues;  defence  planning; 
capability  development;  concept  development  and 
experimentation;  and  current  operations. 

Analytical  Support  to 
Defence  Transformation 

Apr  2005 

055 

The  Studies,  Analysis  and  Simulation  (SAS)  Panel 
conducted  a  three-day  symposium  (SAS-055/RSY)  on 
“Analytical  Support  to  Defence  Transformation”.  It 
intended  to  bridge  the  gap  between  the  operational  and 
research  communities  who  are  responsible  for 
accomplishing  the  Transformation  which  NATO  seeks  and 
the  analytical  community  who  are  charged  to  support  the 
design  and  assessment  of  future  force  effectiveness. 

Five  important  issues  were  discussed: 

1)  The  conditions,  which  motivate  the  need  for,  and  keen 
interest  in  NATO  Transformation; 

2)  Capability  Based  Planning; 

3)  Effects  Based  Operations  and  highlighted  the  need  for 
considering  a  much  broader  set  of  factors  than  those 
considered  by  recent  military  operational  analysis; 

4)  Network-Enabled  Capabilities  (NEC);  and 

5)  Concept  Development  and  Experimentation. 

Operational  Analysis 
Support  to  NATO 
IFOR/SFOR  Operations 

Mar  1999 

003 

The  document  records  the  proceedings  of  a  Symposium 
held  in  June  1997  addressing  actual  Operations  Analysis 
(OA)  support  to  IFOIUSFOR  operations. 

A  total  of  14  papers  were  presented  by  analysts  who  were 
deployed  in  theatre  during  the  deployment  of  IFOR, 
by  analysts  involved  in  logistic  planning,  cost  estimating 
and  C3 1  issues,  and  by  analysts  involved  in  lessons  leamt 
teams.  In  addition  a  forward  looking  presentation  was 
given  concerning  in-theatre  support  to  SFOR. 

2.2  CASE  STUDY  APPROACH 

Given  the  time  constraints  and  demands  for  the  attention  of  senior  NATO  leaders,  the  group’s  consensus  is 
that  the  instructional  material  derived  by  this  study’s  effort  (and  follow-on  contributions)  must  be  able  to  be 
delivered  in  no  more  than  twenty  minutes,  be  executive  in  nature,  and  be  relevant  to  that  specific  NATO 
leader.  Therefore,  we  selected  a  historical  case  study  approach  organized  by  application  areas  for 
instructional  material  development. 
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2.2.1  Relevance 

The  team  identified  six  application  areas  to  consider  for  case  study  generation.  These  application  areas  are 
strategic  planning,  support  of  military  operations,  transforming  armed  forces,  logistics  and  acquisition, 
scheduling,  and  other.  The  “strategic  planning”  application  area  addresses  case  studies  of  national  or 
strategic  importance.  “Support  of  military  operations”  application  area  focuses  on  case  studies  dealing  with 
decisions  or  operations  at  the  operational  level  usually  while  the  operations  are  on-going  or  being  planned. 
The  application  area  titled  “Transforming  Armed  Forces”  deals  with  case  studies  related  to  re-shaping  force 
structures,  processes,  or  bases.  “Logistics  and  acquisition”  application  area  contains  decision-related  cases 
dealing  with  supply,  support,  or  the  procurement  of  forces.  “Scheduling”  deals  with  cases  dealing  with  the 
efficient  employment  of  forces  or  logistics  that  have  an  operational  impact.  The  “other”  application  area 
allows  for  introduction  of  basic  operational  analysis  tools  and  for  case  studies  not  falling  neatly  into  any 
other  application  category  respectively. 

The  objective  of  these  application  areas  is  for  presenters  to  search  under  relevant  topics  that  will  interest  the 
senior  leader  they  intend  to  provide  instruction.  For  example,  if  a  presenter  knows  that  the  audience  will  be 
NATO  Commander  in  charge  of  employment  of  troops,  the  presenter  may  look  in  the  application  areas  of 
Logistics,  Scheduling,  and  Support  for  Military  Operations  to  find  applicable  case  studies.  Once  a  relevant 
case  study  is  found,  the  presenter  may  modify  it  to  suit  the  specific  needs  of  the  flag  or  general  officer. 

2.2.2  Concise  Presentation  at  the  Executive  Level 

As  discussed  in  more  detail  in  Chapter  3,  the  structure  of  the  instructional  presentation  material  is  designed 
to  quickly  demonstrate  the  material’s  relevance  to  the  senior  leader  through  an  actual  case  study  with  a 
decision  challenge  for  a  senior  leader.  The  executive  portion  of  the  material  is  no  more  than  10-15  slides, 
with  the  option  of  more  detailed  analytical  information  following  the  executive  presentation  to  provide  the 
staff  and  analysts  supporting  the  Commander. 

In  addition,  these  case  studies  may  be  useful  in  NATO  Member  Nations’  national  schools  to  provide  studies 
for  specific  operational  analysis  tools.  Table  2-2  shows  a  matrix  that  cross-references  application  areas  with 
specific  operational  analysis  disciplines. 

Table  2-2  is  meant  to  be  a  living  document,  growing  to  accommodate  new  case  studies  after  the  formal 
conclusion  of  SAS-098.  As  operational  analysts  continue  to  contribute  to  NATO  operations,  the  potential 
exists  for  more  case  studies  to  be  added  using  our  study’s  common  template  saved  in  a  location  electronically 
to  make  them  accessible  to  all  NATO  members.  Chapter  3  discusses  this  common  structure  of  an  Instructional 
Module  to  provide  a  summary  of  the  material  and  the  presentation  material  with  a  specific  format. 
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Table  2-2:  Application  Area  and  OA  Tool  Case  Study  Cross  Reference. 


Analytics  and 
Assessment 

Probability 

Statistics 

Data  Analysis 

Optimization 

Decision 
Analysis  and 
Game  Theory 

Soft  OA 

Simulation 

Data 

Visualization 

Strategic 

Planning 

(J5) 

Strategic 

Analysis  and 
EBAO 

Peace  Support 
Operation 

Planning 

Support  for 
Military 

Operations 

(J3/J5) 

Improved 
Situational 
Awareness 
(Correlation  and 
Pattern) 

Improved 

Situational 

Awareness 

(Descriptive 

Statistics) 

General  Officer 
Targeting; 

Persistent  Threat 
Detection  System 

Improved 
Situational 
Awareness 
(Correlation  and 
Pattern) 

Improved 

Situational 

Awareness 

(Descriptive 

Statistics) 

Intra-theater 

Air  Lift 

Peace  Support 
Operation 

Planning 

Chemoff  Faces 

Persistent  Threat 
Detection  System 

Chernoff  Faces 

Transformation 

(J4/J5) 

Visual  Analytics 
in  Operational 
Capability 
Requirements 
Evaluation 

Base 

Realignment  and 
Closure 
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Analytics  and 
Assessment 

Probability 

Statistics 

Data  Analysis 

Optimization 

Decision 
Analysis  and 
Game  Theory 

Soft  OA 

Simulation 

Data 

Visualization 

Logistics  and 
Acquisition 

(J4/J8/J9) 

Cost  Estimation 
for  Major 

Defence  Systems 
(LPD  ship) 

Replenishment  at 
Sea 

DEU  Out 

Planning 

Intra-theater  Lift 

Stockpile 

Planning 

Guidance 

Cost  Estimation 
for  Major 

Defence  Systems 
(LPD  ship) 

Scheduling 

(J3) 

Replenishment 
at  Sea 

DEU  Out 

Planning 

Other 

Operation  Eagle 
Claw 

Operation  Eagle 
Claw 

Probability  and 
Decision  Trees: 
Challenger  Space 
Shuttle 

Probability  and 
Decision  Trees: 
Challenger  Space 
Shuttle 

Expeditionary 
Operations 
(Morphological 
Analysis ) 

Operation  Eagle 
Claw 

Overview 

Making  Better 
Decisions  with 
operational 
Analysis: 

An  Overview 

Making  Better 
Decisions  with 
Operational 
Analysis: 

An  overview 

Making  Better 
Decisions  with 
Operational 
Analysis: 

An  overview 

Making  Better 
Decisions  with 
Operational 
Analysis: 

An  overview 

Making  Better 
Decisions  with 
Operational 
Analysis: 

An  overview 

Making  Better 
Decisions  with 
Operational 
Analysis: 

An  overview 

Making  Better 
Decisions  with 
Operational 
Analysis: 

An  overview 
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Chapter  3  -  INSTRUCTIONAL  MODULE  TEMPLATES 

Ephraim  Robbe,  First  Lieutenant 

Bundeswehr  Planning  Office 
EinsteinstraBe  20 
D-85521  Ottobrunn 
GERMANY 

3.1  NEED  FOR  STANDARD  FORMAT 

The  study  participants  recognized  the  need  for  a  standard  format  to  develop  and  present  case  studies  early  in 
the  SAS-098  effort.  In  addition  to  allowing  a  consistent  development  method  for  this  study,  standard 
instructional  templates  suggest  an  efficient  way  to  capture  new  case  studies  as  they  occur  in  the  future. 
A  case  study  composed  of  three  elements:  the  Instructional  Module  Summary,  the  presentation  with  talking 
notes,  and  any  reference  or  reading  material.  The  Instructional  Module  format  quickly  communicates  to  an 
instructor  the  main  theme  of  the  case  study.  The  presentation  format  quickly  demonstrates  the  relevancy  and 
importance  of  the  case  study  to  a  senior  decision-maker.  This  is  particularly  addressed  in  the  “Overview” 
slide  discussed  below  and  shown  in  Figure  3-1.  The  presentation  format  also  allows  for  more  detail  to  be 
given  to  staff  and/or  analysts  interested  in  the  analytical  tool  used  in  the  case  study.  Finally,  if  there  is 
reference  material  available,  a  third  element  of  the  case  study  can  be  included  for  instructor  and  student  use. 


Overview 


Situation:  Iraq  2006,  perceived  high  number  of  insurgent  attacks 
against  Forward  Operating  Bases  [FOE}  within  30  minutes  after  arrival 
of  a  particular  U.S.  Army  General.  General  believed  he  was  being 
targeted. 


Decision:  Identify  if  General  was  being  selected  for  insurgent  attack 
and  if  so,  discover  means,  which  identified  that  he  was  visiting  the 
FOB. 


Objective:  Decrease  risk  of  General  officer  movements. 

OA  Contribution:  Confirmed  evidence  of  General  officer  targeting 
by  Hypothesis  Testing  (Statistics}  and  made  recommendations  on 
changing  the  movement  procedures  decreasing  the  riskforthe 
General  officer. 


Result  or  outcome:  Reduction  of  insurgent  attacks  against  Generals 


Figure  3-1 :  Summary  Slide. 
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3.2  INSTRUCTIONAL  MODULE  SUMMARY 

The  first  component  -  the  instructional  module  summary  -  consists  of  eleven  parts  which  constitute  a  brief 
overview  of  the  specific  lecture.  It  includes  the  title  of  the  case  study,  the  developer’s  name  and  organization, 
presentation  length  in  time,  learning  objectives  and  corresponding  outcomes,  a  brief  content  outline, 
any  additional  materials  (e.g.  required  pre-readings),  delivery  methods,  and  discussion  questions. 

Table  3-1  illustrates  this  standard  format. 


Table  3-1:  Instructional  Module  Summary  Standard  Format. 


Title 

From  Concept  to  Operational  Method  for  Domestic  Operations,  The  Impact  of 
Strategic  Analysis:  Canada  Command  Effects  Based  Approach  to  Operations 

Faculty 

Name(s) 

Brad  Gladman,  DRDC  CORA,  Ottawa  ON,  Canada 

Length 

60  -  90  minutes 

Learning 

Objectives 

Understand  the  value  of  the  distinct  capability  of  strategic  analysis. 

Appreciate  the  relationship  of  strategic  analysis  to  competent  operational  planning  in 
achieving  the  Commander’s  intent. 

Appreciate  the  need  for  an  appropriate  reporting  relationship,  and  other  essential 
requirements,  in  order  to  realize  the  potential  of  strategic  analysis  in  decision- 
support. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

•  Understand  the  value  of  the  distinct  capability  of  strategic  analysis  in 
decision-support. 

•  Understand  how  strategic  analysis  can  complement  competent  operational 
planning  in  the  development  of  a  clear  operational  method,  which  in  this  case 
was  used  to  change  other  government  department  thinking  about  the  ‘effects’ 
the  military  can  generate  in  various  crises  as  part  of  a  Whole  of  Government 
approach. 

•  Understand  what  strategic  analysis  requires  in  order  to  be  successful  in  any 
organization  it  supports. 

Content 

Outline 

Case  Study:  Canada  Command  Effects  Based  Approach  to  Operations. 

Vague  concept  to  operational  method. 

Concept  tailored  to  domestic/continental  operating  environment  as  part  of  a  Whole 
of  Government  approach. 

Summary  of  strategic  analysis  requirements  for  success  in  any  organization. 
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Materials 

List  Required  Pre-Readings: 

Dr.  Brad  Gladman  and  Dr.  Peter  Archambault,  An  Effects  Based  Approach  to 
Operations  in  the  Domestic  and  Continental  Operating  Environment:  A  Case  for 
Pragmatism  (Ottawa:  DRDC  CORA  Technical  Memorandum  2008-033,  2009). 

Methods 

50  %  Lecture  0  %  Class  Exercises 

_50%  Break  Outs 

Exercise(s) 

Break  Outs 

X  Yes _ No 

#  5 _ participants  in  each  break  out  group 

Discussion 

Questions 

Does  your  organization  have  an  embedded  strategic  analysis  capability? 

In  your  view,  does  it  require  such  a  capability? 

In  your  experience,  in  what  instances  would  your  organization  have  benefitted  from 
strategic  analysis  for  decision-support? 

How  should  this  capability  be  structured  and  how  should  it  report  to  the  senior  level 
of  your  organization? 

If  this  capability  did  not  report  to  the  highest  level  of  your  organization, 
was  its  impact  impaired?  If  so,  how? 

3.3  PRESENTATION 

The  second  component  -  the  case  study  presentation  -  has  two  main  parts:  a  ten  to  twenty  minute  summary 
for  a  general  officer,  flag  officer,  or  executive  decision-makers  and  a  second  part  for  more  junior  staff 
officers  or  analysts  wishing  more  detail.  The  specific  power  point  template  for  presentation  generation 
allows  for  modification  by  future  lecturers. 

After  the  title  slide,  there  is  a  summary  slide  to  capture  an  executive’s  attention  and  show  immediate 
relevance,  it  includes  information  on  the  situation,  decision  to  be  made,  objective,  operational  analysis’ 
contribution,  and  the  result  or  outcome.  Figure  3-1  shows  the  summary  slide. 

Five  to  ten  slides  follow  this  summary  slide  expanding  these  areas.  Figure  3-2  illustrates  how  the  relevant 
parts  can  be  explicated. 
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Approach 


Experience  of  the  General:  his  visits  triggered 
the  attacks  on  FOBs 

Hypothesis:  it  is  “deliberate”  and  not 
“random”  that  there  are  more  attacks  on 
FOBs  after  helo  landing  with  Generals  sitting 
in  the  helo! 

Testing  by  the  OA:  seeking  evidence  by 
collecting  and  analyzing  data  ->  providing 
evidence  for  hypothesis 


Figure  3-2:  Adding  Specifics  to  the  Case  Study. 


A  clear  break  occurs  where  even  more  detailed  information  is  provided  for  staff  officers  and  analysts. 
Figure  3-3  gives  an  example  of  detailed  information  on  the  conducted  analysis. 


Details 


•  Underling  probability  distribution  for  this  analysis  was 
the  binomial,  or  how  many  “successes”  will  we  have 
from  so  many  trials. 

•  This  analysis  involved  hypothesis  testing  with  HO 
being  the  General’  helo  landing  is  just  another 
random  event  with  probability  of  .2  that  it  will  trigger 
an  indirect  attack  on  the  FOB. 

•  The  question  then,  is  what  are  the  chances  of 
experiencing  ten  or  more  attacks  in  20  trials  if  each 
“trial”  has  a  probability  of  .2  of  trigging  an  attack. 

•  The  answer  is  less  probability  than  rolling  three  sixes 
with  three  dice:  .3%,  which  is  how  the  analyst 
describe  this  to  the  general. 


Figure  3-3:  Detailed  Information  for  Staff  Officers  and  Analysts. 
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Additionally,  every  presentation  is  -  where  necessary  and  appropriate  -  annotated  with  additional 
information  for  the  lecturer.  This  enables  future  presenters  to  conduct  lectures  regardless  of  their  prior 
knowledge  on  a  specific  case  study. 


3.4  BACKGROUND  READING  MATERIAL 

The  third  component  -  the  background  reading  material  -  consists  of  any  reading  material  deemed  suited  to 
facilitate  the  understanding  of  a  topic  (e.g.  required  pre -readings).  There  is  no  standard  format  for  the 
presentation  of  this  material. 

All  three  components  combined  constitute  an  instructional  module.  Chapter  4  discusses  the  archiving  of 
these  modules  as  well  as  alternative  delivery  venues. 
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Chapter  4  -  DELIVERY  AND  ARCHIVING  RECOMMENDATIONS 

Professor  of  Practice,  Jeff  Kline,  Captain,  USN  (ret) 

Naval  Postgraduate  School 
1411  Cunningham  Road,  Room  236 
Monterey,  CA  93943 
UNITED  STATES 


4.1  INTRODUCTION 

This  study’s  focus  was  to  select  topic  areas,  create  instructional  templates,  and  develop  relevant  case  study 
instructional  modules  for  senior  officers  to  appreciate  how  operational  analysis  can  assist  in  their  decision¬ 
making  challenges.  Once  developed,  however,  an  accessible  archiving  medium,  a  venue  for  case  study 
delivery,  and  a  method  for  updating  or  creating  new  case  studies  past  this  particular  study’s  effort  are  needed 
for  NATO-wide  use.  This  section  presents  several  alternatives  for  consideration  and  discusses  specific 
actions  already  taken  to  share  this  study’s  production. 


4.2  ARCHIVING  CASE  STUDIES  ON  THE  WEB 

To  make  the  case  studies  developed  for  this  study  accessible  to  all  NATO  staff  and  instructors  a  web-based 
approach  is  recommended.  The  case  study  presentations,  their  instructional  modules,  and  any  recommend 
readings  can  be  stored  electronically  and  assessable  to  all  NATO  Member  Nations  for  review,  download, 
and  use.  As  such,  the  case  studies  may  find  their  way  into  national  and  NATO  Education  and  Training 
curricula.  For  NATO  this  means  that  the  case  studies  need  to  be  incorporated  into  the  NATO  Education  and 
Training  Opportunity  Catalogue  (ETOC)  and  thus  advertise  and  make  the  material  available  to  NATO  and 
non-NATO  Nations  alike.  A  second  step  would  be  to  incorporate  the  case  studies  into  formal  national  and 
NATO  courses.  For  NATO  this  would  mean  that  the  case  studies  would  be  accredited  formally  and  thus  can 
be  offered  as  material  in  NATO  courses  or  form  their  own  course.  For  now  SAS-098  participants  leveraged 
Allied  Command  Transformation’s  operational  analysis  web  site  to  initially  display  the  study’s  finished 
products.  This  site  may  be  found  at  https://transnet.act.nato.int/WISE/NATOAnalys  and  is  displayed  in 
Figure  4-1. 


File  Edit  View  Favorites  Tools  Help 
x  %  Convert  »  K  Select 


X  p  [  |  *  converter  Download  for  Desktop  Download  for  iPhone  #  Advanced  Download  More* 


Figure  4-1:  Allied  Command  Transformation  (ACT)  NATO  Analysis  Web  Site. 
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It  is  envisioned  that  an  instructor  from  a  Member  Nation’s  senior  military  schools  or  from  a  NATO  school 
will  review  the  various  case  studies  to  find  one  relevant  to  the  senior  officer  that  will  receive  the  instruction, 
download  the  instructional  material,  modify  it  to  the  specific  audience,  and  use  it  in  presentation.  As  the 
analytical  community  experiences  new  situations  where  analysis  is  effectively  used  in  operational  decisions, 
new  case  studies  are  developed  by  NATO  analysts  using  the  standard  templates  and  submitted  to  the  ACT 
website  to  share  with  others.  Again,  however,  in  the  future  this  role  may  also  be  filled  by  a  NATO  School, 
or  a  Member  Nation’s  military  school  website.  For  example,  a  short  summary  of  the  OA  executive  case 
study  program  may  be  provided  in  the  NATO’s  Education  and  Training  Opportunity  Catalogue  (ETOC). 


4.3  ALTERNATIVE  VENUES  FOR  CASE  STUDY  DELIVERY 

Several  venues  are  recommended  for  case  study  delivery.  These  include  STO  lecture  series,  existing 
Member  Nation  and  NATO  courses  and  seminars,  NATO  exercises  or  war  games,  and  during  normal  staff 
operations. 

STO  Lecture  Series  provide  an  opportunity  to  present  case  studies  to  NATO  senior  officers  and  NATO  staff 
members  as  part  of  an  existing  NATO  Lecture  Series  program.  The  STO  SAS  Panel  is  considering  a 
Technical  Activity  Proposal  (TAP)  for  a  STO  Lecture  Series  to  present  several  case  studies  from  this  study 
to  Allied  Command  Transformation,  SHAPE,  and  the  NATO  Defence  College.  If  executed,  the  Lecture 
Series  will  serve  two  purposes.  The  first  is  to  deliver  particular  case  studies  of  interest  and  operationalize  this 
study,  and  the  second  is  to  inform  NATO  analysts  of  the  existence  of  the  case  studies  for  their  use. 

Existing  formal  courses  and  seminars  for  senior  officers  in  NATO  and  Member  Nation  defence  schools  may 
utilize  these  case  studies  for  examples  in  normal  course  work.  The  publication  of  the  SAS-098  study  will  aid 
in  informing  the  NATO  community’s  instructors  of  the  case  studies’  availability,  but  annual  NATO 
analytical  and  staff  conferences  provide  other  method  to  disseminate  information  on  their  existence. 
In  addition,  instructors  will  be  invited  to  submit  their  own  case  studies  modified  in  the  standard  format 
discussed  in  Chapter  3. 

Flag  and  general  officer  seminars  inside  NATO  exercises  and  war  games  provide  another  opportunity  for 
case  study  presentation,  particularly  if  the  case  study  relates  to  that  particular  exercise’s  or  game’s  objective. 
Exercise  designers  will  use  the  case  study  matrix  in  Table  2-2  to  select  applicable  case  studies  then  schedule 
presentation  time  to  the  senior  officers.  If  the  case  study’s  particular  operational  analytical  tool  is  made 
available  to  assist  in  the  exercise  it  will  be  reinforcing  for  the  case  study  and  provide  increased  appreciation 
for  the  analytical  staff. 

NATO  staffs  analysts  may  also  use  a  particular  case  study  to  demonstrate  a  tool  or  analytical  method  which 
they  wish  to  apply  to  a  similar  decision  or  upcoming  operation.  This  meets  the  original  intent  of  this  study’s 
charter  by  advancing  the  general  officer’s  knowledge  on  analytical  capabilities  while  inspiring  the  use  of 
analysis  in  an  operation.  NATO  staff  analysts  are  also  welcome  to  submit  new  case  studies  as  they  apply 
their  craft  to  aid  in  decision-making  and  operations. 
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Appendix  1  -  FULL  CONTRIBUTOR  LIST 

Included  below  are  the  names  of  all  those  who  contributed  to  this  study,  regardless  of  official  membership 
status. 


Name 

Country 

Aplak,  Hakan  Soner 

Turkey 

Bourdon,  Sean 

Canada 

Buch,  Heinrich 

Germany 

Cameron,  Fred 

Canada 

Dell,  Robert  (Co-Chair) 

USA 

De  Nijs,  Johannes  (Co-Chair) 

NATO  ACT 

Dompke,  Uwe 

Germany 

Dickson,  Paul 

Canada 

Gladman,  Brad 

Canada 

Grainger,  Julius 

NATO  NCIA 

Kline,  Jeffrey  (Co-Chair) 

USA 

Kose,  Erkan 

Turkey 

Leopold,  Armin 

Germany 

McNaught,  Ken 

United  Kingdom 

Monsuur,  Herman 

Netherlands 

Pietzschmann,  Harold 

Germany 

Robbe,  Ephraim 

Germany 

Sok,  Sam 

USA 

Trehame,  James 

USA 

Utterbeeck,  Filip  Van 

Belgium 
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Country 

Wellbrink,  Joerg 

Germany 

Wetschoreck,  Hans-Hermann 

Germany 

Wittmann,  Christian 

Germany 

Zuna,  Pavel 

Czech  Republic 

Administrative  and  Staff  Support 

Carroll,  Chris  (RTO  CSO) 

USA 

Englehom,  Lyla  (NPS) 

USA 

Landovsky,  Roman 

Czech  Republic 

Tahar,  Rina 

France 

A1  -2 


STO-TR-SAS-098 


NATO 

OTAN 


organization 


Appendix  2  -  TERMS  OF  REFERENCE  (TOR) 


UNCLASSIFIED /UNLIMITED 

Terms  Of  Reference  (TOR) 


S AS- 098,  RTG-043 
on 

Operations  Research/Operations  Analysis  Orientation  Course  Curriculum  for  NATO  Nations 


I.  Origin 

A.  Background 

As  aprofessional  group,  the  military  personnel  of  NATO  nations  receive  high-level,  quality  education  from  a  wide  range  of 
academic  disciplines  throughout  their  career.  The  need  for  high-quality  analysis  to  support  decision  making  in  military 
acquisition,  in  force  development,  in  force  employment  and  in  management  of  the  national  military  and  NATO  organizations 
compelling  that  continued  emphasis  be  placed  on  scientific  rigor  and  academic  qualities.  While  in  the  academic  world,  the 
research  and  development  organizations,  and  the  science  labs,  these  attitudes  are  reinforced  daily,  the  work  environment  of  the 
military  staffs  is  focused  more  on  the  results,  achievements  and  the  bottom-line,  and  therefore  can  lead  the  military  away  from 
scientific  methods. 

Military  Operations  Research  (OR)  or  Operational  Analysis  (OA)  is  defined  as  the  application  of  scientific  methods  to  assist 
executive  decision  makers  and  as  such  is  the  analytical  study  of  military  problems  undertaken  to  provide  responsible 
commanders  and  staff  agencies  with  scientific  based  reasoning  for  decisions  on  actions  to  improve  military  operations. 
OR/OA  provides  support  for  decisions  in  most  of  the  military  management  domains,  and  OR/OA  presents  itself  as  a  powerful 
tool  to  help  improve  the  quality  of  decision  making  by  illuminating  key  issues,  assumptions  and  sources  of  information. 
OR/OA  is  an  academic  discipline  whose  roots  are  in  application  of  hard  and  soft  science  methods  in  military  decision  making 
during  World  War'  II.  Many  NATO  nations  have  a  rich  history  of  OR/OA  and  currently  employ  it  extensively  throughout  their 
organizations.  These  nations  also  have  many  OR/OA  educational  opportunities  for  their  military  personnel.  However,  such 
opportunities  are  not  universal.  Shar  ing  experiences  and  developing  educational  opportunities  for  NATO  decision  makers  and 
their  staff  will  enhance  OR/OA  use  throughout  NATO. 

B .  Ju stiflcation  (Relevance  for  NAT  O) 

The  need  to  use  OR/OA  in  staff  and  commanders  decision  making  is  easily  overlooked  in  todays  busy  schedules,  however, 
OR/OA  when  applied  successfully,  will  almost  always  result  in  savings  and  increased  efficiency  or  effectiveness  or  both. 
Appreciation  of  the  contribution  of  OR/OA  by  NATOs  military  may  therefore  be  necessary. 

II.  Objectives 

The  objective  of  this  effort  is  the  design  of  a  set  of  instructional  modules  to  demonstrate  to  the  non-analyst  the  value  provided 
by  various  analytical  tools  in  operational  decision  making.  The  material  will  serve  as  source  material  for  integration  into  the 
continuing  education  curriculums  ofNATO  and  national  military  staffs.  For  example,  these  modules  may  be  used  to  build  a 
tailored  week-long  course  for  senior  officers  (up  to  OF-5)  or  in  condensed  form  serve  as  an  abbreviated  information  session 
for  flag  and  general  officers.  Nations  will  use  the  modules  to  tailor  their  own  courses.  NATO  nations  with  little  or  no  OR/OA 
heritage  will  be  provided  with  material  they  can  use  to  raise  the  awareness  among  their  military  professionals. 

Tli is  proposal  attempts  to  gather  national  OR/OA  experts,  preferably  with  teaching  experiences  in  military  OR/OA,  to  propose 
the  OR/OA  curriculum  modules  for  such  a  course  and  design  it  so  that  it  can  be  used  and  offered  in  national  and  NATO 
military  academies,  schools  and  colleges.  It  may  be  necessary  to  survey  national  defence  education  institutes  to  understand  the 
level  of  OR/OA  teaching  and  determine  for  which  topics  and  to  which  depth  level  these  lessons  need  to  be  developed.  The 
group  will  develop  learning  objectives,  module  outlines,  framework  lessons  plans  and  applicable  class  material  for  each 
module.  The  group  will  gather  and  build  from  existing  material  from  national  OR/OA  courses  and  build  the  modules  for  use 
in  NATO  courses  or  in  national  curriculums.  This  course  material  could  be  further  tailored  by  the  nations  to  serve  national 
purposes. 
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III.  Resources 

A.  Membership 


UNCLASSIFIED /UNLIMITED 

Terms  Of  Reference  (TOR) 


Members  of  the  Task  Group  should  have  a  strong  background  in  OA/OR,  with  a  background  in  OA/OR  education  desired. 
Belgium,  Canada,  Czech  Republic,  Germany,  Netherlands,  United  Kingdom,  and  the  United  States  have  expressed  a 
willingness  to  participate  in  the  work. 


CAPT  (USN,  retired)  Jeff  Kline  of  the  United  States  is  willing  to  lead  the  Task  Group. 

Co-Chair  :  Mr.  Johannes  DENIJS  Netherlands 

Co-Chair  :  Dr.  Robert  DELL  United  States 

Co-Chair  :  Capt  USN  (ret)  Jeffrey  E.  KLINE  United  States 

Lead  Nation:  United  States 

Nations  and  Bodies  Really  Participating:  ACT,  Czech  Republic,  Germany,  Netherlands,  United  States 

B.  National  And/Or  NATO  Resources  Needed  : 

It  is  expected  that  the  RTG  will  meet  at  least  twice  a  year  over  its  duration.  Nations  should  be  prepared  to  dedicate  funds 
necessary  to  support  the  required  travel  of  the  RTG  members.  However,  if  formed,  this  RTG  will  explore  the  maximum  use  of 
VTC  and  web  technologies  for  coordination  and  work  to  limit  travel  requirements.  In  addition,  nations  should  allocate 
sufficient  time  within  their  respective  research  work  programs  to  allow  assigned  researchers  to  perform  the  necessary 
activities  of  the  RTG. 


C.  RTA  resources  needed 

RTA  meeting  facilities  will  be  used  for  the  initial  meeting  of  the  RTG. 

IV.  Security  Classification  Level 

The  security  level  will  be  Unclassified/Unlimited 

V.  Participation  By  Partner  Nations 

Invitation  to  participate  will  be  extended  to  all  NATO  nations,  as  well  as  NATO  Partner  nations, 
see  Membership 

VI.  Liaison 

The  Task  Group  will  seek  cooperation  with  those  organizations  whose  activities  are  connected  to  this  topic. 
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Operations  Research  /  Operations  Analysis 
Orientation  Course  Curriculum 
for  NATO  Nations 

ANNEX 

Application  Areas  with  Supporting  Curriculum 


This  annex  document  is  intended  to  accompany  the  full  report. 
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A.O  ORIENTATION 

The  developed  curriculum  is  separated  out  into  seven  pre-defmed  Application  Areas: 

1)  Support  for  Military  Operations; 

2)  Strategic  Planning; 

3)  Transformation; 

4)  Logistics  and  Acquisition; 

5)  Scheduling; 

6)  Other;  and 

7)  Overview. 

Within  each  of  these  Application  Areas,  the  case  studies  fall  into  several  distinct  Operational  Analysis 
Discipline  Areas.  Some  case  studies  fall  in  two  of  these  discipline  areas.  These  include: 

a)  Analytics  and  Assessment; 

b)  Probability  Statistics  Data  Analysis; 

c)  Optimization; 

d)  Decision  Analysis  and  Game  Theory; 

e)  Soft  OA; 

f)  Simulation;  and 

g)  Data  Visualization. 

For  each  developed  case  study,  this  annex  includes  the  module  overview  followed  by  the  slides  and 
instructor  notes  for  each  case  study  organized  within  the  pre-defined  Application  Areas.  Note  that  some  case 
studies  are  included  in  more  than  one  Application  Area. 
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A.l  OVERVIEW 

A.1.1  Making  Better  Decisions  with  Operational  Analysis 


INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Making  Better  Decisions  with  Operational  Analysis 

Faculty 

Name(s) 

Jeff  Kline,  Naval  Postgraduate  School,  Monterey,  California,  USA 

Length 

15-30  minutes 

Learning 

Objectives 

Provide  an  appreciation  for  how  operational  analysis  can  aid  in  decision¬ 
making. 

Provide  a  general  definition  of  operational  analysis,  its  purpose,  and  history. 

Give  some  example  tools  used  by  analyst  to  aid  in  decisions. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Better  utilize  the  analysts  on  a  NATO  staff  to  aid  in  operations  and  decisions 
in  planning. 

Content 

Outline 

Operational  Analysis  defined. 

Origins  of  Operational  Analysis. 

Contributions  of  OA  to  decisions. 

Examples  of  OA  approaches  and  tools. 

Case  study  matrix  for  how  it  has  been  used. 

Materials 

List  Required  Pre-Readings: 

None. 

Methods 

100%  Lecture  0%  Class  Exercises. 

%  Break  Outs. 

Exercise(s) 

Seminar  participants  will  be  ask  to  discuss  relevancy  of  approaches  to  their 
past  decision  challenges. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

What  are  your  experiences  with  manual  scheduling? 

How  might  optimization  improve  that  process? 
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Making  better  decisions  with 
operational  analysis 


An  Introduction 

I7.1Q.I2 


This  presentation  is  an  overview  of  operational  analysis  and  some  of  its  tools  to  aid  in  flag 
and  general  officer  decision  making.  It  is  not  meant  to  be  a  comprehensive  inventory  of 
the  art  and  tools  associated  with  operations  research,  operational  analysis,  and  analytics. 
Nor  are  the  definitions  of  methods  academically  rigorous.  It's  purpose  is  to  give  a  brief 
summary  on  how  an  operations  analyst  can  aid  a  senior  decision  maker  in  making  better 
decisions  in  language  a  senior  decision  maker  may  understand. 


Overview: 

Operational  Analysis  (OA) 


Operational  Analysis  (OA)  is  the 
science  of  helping  people  and 
organizations  make  better 
decisions 


OA  is  a  way  of  thinking  and  applying  a 
rational  process  in: 

■  Decision  making 
•  Choices  of  action 

■  Probability  and  impact  of  risks 

I7.1QJ;  J  * 


More  formally,  operational  analysis  is  the  development  arid  application  of  mathematical 
models,  statistical  analyses,  simulations,  analytical  reasoning,  analytics  and  common  sense 
to  the  understanding  and  improvement  of  real-world  operations.  Just  as  important,  it  is  a 
way  of  thinking  a  nd  applying  a  technical  and  rational  process  to  understand  problems, 
choices,  outcomes,  and  risk. 
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EEjg  Origins  of  Operational 

if* 

Analysis 

*  Physicists,  mathematicians,  and  other 
disciplines  were  called  upon  in  World  War  II 
to  work  finding  solutions  to  their  country’s 
operational  and  tactical  challenges 

♦  Integrating  radar  and  fighters  through  a 
command  and  control  center  during  the 
Battle  of  Britain  is  one  example. 
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OR  marks  its  origins  as  a  discipline  from  the  combined  efforts  of  scientists  to  help  tackle 
their  countries'  operational  and  tactic  issues  during  World  War  II. 


The  British  applied  OR  to  help  design  an  integrated  air  defense  system  during  the  Battle  of 
Britain,  while  both  the  Americans  and  Germans  applied  these  techniques  in  the  Battle  of 
the  Atlantic. 


OA  can  contribute  to  every 
level  of  decision 


*  Strategic  Decisions:  National  Policy  analysis, 
resource  allocation,  force  composition  and 
modernization 


■  Operational  Decisions:  Force  allocation, 
human  resource  planning,  logistics,  flight 
operations  scheduling 


*  Tactical  Decisions:  Targeting,  engagement 
tactics,  optimal  routing,  and  search  tactics 


Today,  Operations  Research  and  operational  analysts  has  been  applied  at  every  level  of 
decision  making,  from  International  policy  creation  to  routing  of  search  assets  for  downed 
submarines. 
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^  Approaches 

*  Analytics  and  assessments:  Aids  in  selecting 
operational  objectives  and  metrics  to 
monitor  progress  toward  those  goals. 


-  "  ■  *  '  ~ 

iZ 

1. 

^1 

“  V™  “  “  “  "  “ 

•  Probability  and  Statistics:  Helps  to  bound 
uncertainty  by  understanding  the 
probabilities  of  risk  and  of  success  In  courses 
of  action 


Analytics  and  assessments  can  be  directly  Integrated  into  a  commander's  planning  process 
to  help  identify  measurable  objectives  from  overall  operational  goals  and  select  metrics  to 
monitor  progress  toward  those  goals.  Many  staffs  now  have  net  assesssment  sections. 
Integrating  operational  analysts  with  these  sections  and  planning  efforts  will  increase  rigor 
in  the  quantitative  monitor  of  mission  achievement  and  improved  situational  awareness. 

Probability  and  statistics  represent  a  host  of  tools  to  help  bound  uncertainly  and  quantify 
risk  in  decision  making.  These  techniques  are  used  in  cost  estimation,  risk  assessment, 
search  tactics,  process  modeling,  hypthos is  testing,  regression  modelling,  survey  and  data 
analysis.  These  techniques  have  been  used  in  such  diverse  areas  as  to  confirm  a  general 
officer's  hello  was  being  targeted  to  understanding  how  much  is  required  fora  contested 
rescue  mission. 
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B  Approaches 

*  Optimization:  Methods  to  improve 
processes  through  minimization  of  costs, 
maximization  of  efficiency,  or  bettering  of 
other  relevant  measures. 


*  Simulation:  Use  of  models  to  explore  impact 
of  variability  in  initial  operational  conditions 
toward  goals  and  objectives. 


Optimization  is  a  form  of  modeling  that  provides  best  solutions  dependent  on  objectives 
and  contraints.  It  is  used  in  such  areas  as  schedule  aircraft  lift  to  stationing  ballistic  missile 
defense  sites. 

Simulation  can  be  used  to  explore  and  Identify  the  important  initial  conditions  of  a  conflict 
(i.e.  Sensor  location  and  performance,  weather,  weapon  location  and  performance)  to 
understand  risk  assocated  with  various  courses  of  action.  It  has  been  used  in  everything  for 
force  structure  planning  to  tactical  evaluation. 


H  Approaches 

*  Decision  Trees,  Influence  diagraming  and 
Branch  graphing:  Aids  in  bounding  and 
categorizing  decision  space,  understanding 
risk,  and  defining  results  based  on 
conditional  decisions  and  events 


Decision  trees,  influence  diagraming  and  branch  graphicing  are  used  to  understand  the 
problem,  catagorize  and  deconstruct  a  complex  issue,  and  determine  probabilistic  risk. 
These  techniques  have  been  used  in  analyzing  accidents  like  the  space  shuttle  events  to 
determining  courses  of  action  in  operations. 
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Approaches 


*  Morphological  Analysis:  Helps  in 
understanding  relationships,  dimensions  and 
configurations  in  a  complex  problem 


*  Data  Visualization:  Techniques  to  aid  in 
decision  making  by  graphing  and 
categorizing  quantitative  and  non- 
quantitative  data 


Morphological  analysis  can  help  in  deconstructing  qualitative  complex  issues  to  help  in 
understanding  dimensions  to  the  problem.  An  example  is  creating  a  precise  defiination  for 
expeditionary  operations  for  NATO  based  on  mission  requirements  and  description. 

Data  visualization  aids  decision  makers  and  planners  on  understanding  relationships  and 
assessments  of  quantitative  and  qualitative  issues.  A  recent  exam  pie  is  of  its  use  is  to 
assess  military  commands4  abilities  to  carry  out  specific  NATO  tasking. 


Developed  Case  Study  Examples 
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Brief  case  study  presentations  have  been  created  to  demonstrate  specific  use  of  these  tools 
in  real  world  examples  where  flag  or  general  officer  decisions  were  involved.  This  matrix  is 
an  illustrative  set  of  these  case  studies. 
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B  Summary 

*  Operational  analysts  apply  rational 
processes  to  aid  in  decision  making  and 
monitoring  operations* 

*  To  best  utilize  your  OAs  to  aid  in  decision 
making,  they  need  access  to  you*  the 
decision  maker, 

*  The  OA  brings  a  variety  of  tools  like 
optimization,  simulation,  statistics,  and 
assessment  skills  to  help  you  with  evidence 
based  decision  making* 
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A.1.2  Case  Study  Collection  Overview 


Analytics  and 
Assessment 

Probability 

Statistics 

Data  Analysis 

Optimization 

Decision  Analysis 
and  Game  Theory 

Soft  OA 

Simulation 

Data  Visualization 

Strategic 

Planning 

(J5) 

Strategic  Analysis 
and  EBAO 

Peace  Support 
Operation  Planning 

Support  for 

Military 

Operations 

(J3/J5) 

Improved  Situational 
Awareness 
(Correlation  and 
Pattern) 

Improved  Situational 

Awareness 

(Descriptive 

Statistics) 

General  Officer 
Targeting; 

Persistent  Threat 
Detection  System 

Improved 
Situational 
Awareness 
(Correlation  and 
Pattern) 

Improved 

Situational 

Awareness 

(Descriptive 

Statistics) 

Intra-theater  Air 
Lift 

Peace  Support 
Operation  Planning 

Chemoff  Faces 

Persistent  Threat 
Detection  System 

Chernoff  Faces 

Transformation 

(J4/J5) 

Visual  Analytics  in 
Operational 

Capability 

Requirements 

Evaluation 

Base  Realignment 
and  Closure 
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Analytics  and 
Assessment 

Probability 

Statistics 

Data  Analysis 

Optimization 

Decision  Analysis 
and  Game  Theory 

Soft  OA 

Simulation 

Data  Visualization 

Logistics  and 
Acquisition 

(J4/J8/J9) 

Cost  Estimation 
for  Major 

Defence  Systems 
(LPD  ship) 

Replenishment  at 
Sea 

DEU  Out 

Planning 

Intra-theater  Lift 

Stockpile 

Planning 

Guidance 

Cost  Estimation  for 
Major  Defence 
Systems  (LPD 
ship) 

Scheduling 

(J3) 

Replenishment  at 
Sea 

DEU  Out 

Planning 

Other 

Operation  Eagle  Claw 

Operation  Eagle 
Claw 

Probability  and 
Decision  Trees: 
Challenger  Space 
Shuttle 

Probability  and 
Decision  Trees: 
Challenger  Space 
Shuttle 

Expeditionary 
Operations 
(Morphological 
Analysis ) 

Operation 

Eagle  Claw 

Overview 

Making  Better 
Decisions  with 
operational  Analysis: 
An  Overview 

Making  Better 
Decisions  with 
Operational 
Analysis:  An 
overview 

Making  Better 
Decisions  with 
Operational 
Analysis:  An 
overview 

Making  Better 
Decisions  with 
Operational 
Analysis:  An 
overview 

Making  Better 
Decisions  with 
Operational 
Analysis:  An 
overview 

Making  Better 
Decisions  with 
Operational 
Analysis:  An 
overview 

Making  Better 
Decisions  with 
Operational 

Analysis:  An 
overview 
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A.2  SUPPORT  FOR  MILITARY  OPERATIONS 
A.2.1  General  Officer  Targeting 


INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Decrease  Risk  for  General  Officers  Being  Targeted  in  Operations:  A  Case 
Study  in  Hypothesis  Testing 

Faculty 

Name(s) 

Bundeswehr  Planning  Office,  Ottobrunn,  DEU 

Length 

20-30  minutes 

Learning 

Objectives 

Understand  that  OA  methods  can  be  used  to  test  critical  assumptions 
underlying  the  decision-making  process. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Recognize  similar  applications  where  an  analysis  by  OA  staff  could  support 
the  decision-making  process  concerning  critical  security  issues. 

Content 

Outline 

Case  Study:  Decrease  Risk  for  General  Officers  Being  Targeted  in 

Operations. 

Materials 

None. 

Methods 

100%  Lecture  0%  Class  Exercises. 

%  Break  Outs. 

Exercise(s) 

None. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

Did  you  have  a  similar  case  in  the  past? 

Can  you  think  of  a  current  case  in  your  area  of  responsibility? 
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Decrease  Risk  for 
General  Officers 
being  targeted  in  Operations 


A  Case  Study  in  Hypothesis  Testing 


liM  12 


The  case  study  addresses  the  concern  of  a  general  officer  during  Iraq  operations.  He  felt 
forward  operating  bases  were  being  attacked  with  more  frequency  when  he  was  present. 
These  attacks  occurred  while  conducting  routine  visits  via  helicopter. 


Overview 


Situolion:  Iraq  2006.  perceived  high  number  of  insurgent  a  backs 
against  Forward  Operating  Boses  {FOB)  within  30  minutes  offer  arrival 
of  a  particular  U.S,  Army  General  General  believed  he  was  being 
torgefed 

Decision:  Identify  it  General  was  being  selected  for  insurgent  attack 
and  if  so,  discover  means,  which  identified  that  he  was  visiting  the 
FOB. 

Objective:  Decrease  risk  of  General  officer  movement 

OA  Contribution:  Confirmed  evidence  of  Genera!  officer  targeting 
by  Hypothesis  Testing  (Sfatisricsj  and  mode  recommendations  on 
changing  the  movement  procedures  decreasing  the  risk  for  the 
General  officer. 


Result  or  outcome:  deduction  of  insurgent  attacks  against  Generals 

I2JCB.  12  2 


In  this  situation,  the  general  turned  to  his  analysts  to  understand  if  he  was  being  targeted 
by  insurgents.  The  analysts  begin  to  look  at  the  number  of  imorter  attacks  occurmg  at  all 
bases  compared  to  attacks  with  the  general  was  present. 


organization 
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B  Situation 

*  Iraq  Forward  Operating  Bases  (FOB) 
received  indirect  fire  frequently,  General 
officer  visit  or  not. 

*  A  particular  U.$,  Army  General  had  made 
twenty  helo  trips  and  in  ten  of  these,  the  FOB 
being  visited  received  indirect  fire  within 
thirty  minutes  of  his  helo  landing. 


All  forward  operating  bases  received  indirect  fire,  but  in  the  general's  case,  in  twenty  visits 
to  the  bases,  ten  were  attacked  within  30  minutes  of  his  landing. 


B  Approach 

•  Experience  of  the  General:  his  visits  triggered 
the  attacks  on  FOBs 

♦  Hypothesis:  it  is  ‘'deliberate”  and  not 
"random”  that  there  are  more  attacks  on 
FOBs  after  helo  landing  with  Generals  sitting 
in  the  helo! 

*  Testing  by  the  OA:  seeking  evidence  by 
collecting  and  analyzing  data  ->  providing 
evidence  for  hypothesis 


The  general's  personal  experience  led  him  to  believe  he  was  being  targeted,,  but  was  this 
different  than  any  normal  attack  schedule  on  any  base  regardless  of  his  presence? 

By  collecting  the  data  across  all  forward  bases,  the  operational  analysts  were  able  to 
compare  attacks  that  occurred  in  all  cases  with  attacks  occuring  only  when  the  general  was 
present. 
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B  OA  Contribution 


*  Collecting  the  Data 

o  During  the  time  period  of  interest,  how  many 
indirect  attacks  occurred  on  FOBs?  Of  these, 
how  many  attacks  were  there  within  thirty 
minutes  of  a  helo  landing? 

o  Result:  Based  on  the  data,  any  '‘random”  Helo 
landing  triggering  attack  on  a  FOB  was  20%. 

*  “Generals  Experience"  (Data) 

o  Based  on  the  data,  the  General's  Helo  landing 
triggered  a  FOB  attack  1 0  out  of  20  times  (i.e. 
50%} 


Using  a  helo  landing  as  a  trigger  point,  it  was  discovered  that  an  attack  followed  a  helo 
landing  about  20%  of  the  time  in  all  bases.  However,  if  was  the  general's  hello,  an  attacked 
followed  50%  of  thetime. 


OA  Contribution 
Providing  Evidence 


*  Hypothesis  Testing 

o  Chance  for  50%  of  FOB 
attacks  after  a  helo 
landing  (General  officer 
movements}  is  only  0.3% 


mam 

3  times 


■  Ida  -JT- 1  4  li  n  m  -m  g|  r 


CONCLUSION:  Data  analysis  supports  the 
Generals  hypothesis 
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The  foil  owing  conclusion  can  then  be  made.  If  the  general  was  NOT  being  targeted,  the 
chances  of  him  experiencing  10  out  of  20  attacks  just  on  random  attacks  is  the  same  as 
rolling  three  sixes.  Therefore,,  the  data  supports  the  General's  impression  that  he  is  being 
targeted. 
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Follow-on  actions 
supported  by  OA 


*  Several  indicators  were  discovered  which 
could  have  indicated  General  officer 
movements: 

o  Clear  communication  references 
a  General  officer  heio  markings 


*  Once  these  indicators  were  removed, 
the  risk  of  insurgent  attacks  for  the 
General  officer  movements  decreased 
from  50%  to  20%. 


As  a  follow-on,  it  was  discovered  that  the  general's  pilot  used  his  same  call  sign  to  radio  the 
general's  arrival  as  they  approached  the  forward  operating  base.  These  transmission  were 
not  secure.  In  addition,  the  general's  hello  had  unique  markings  that  once  removed,  led  to 
decrease  in  the  number  of  attacks  experienced  after  his  landings. 


B  Summary 

*  Without  evidence  provided  by  OA  there 
might  have  been  no  thorough  analysis  of 
the  movement  pattern 

*  To  best  utilize  your  OAs  to  aid  in  decision 
making,  they  need  access  to  you,  the 
decision  maker. 

*  The  OA  brings  a  variety  of  fools  like  optimiza¬ 
tion,  simulation,  statistics  fe.g.  hypothesis 
testing),  and  assessment  skills  to  help  you 
with  evidence  based  decision  making. 


The  contribution  the  operational  analyst  was  to  bring  data  a ma lysis  skills  to  determine  if  the 
general's  concerns  were  valid.  This  ts  just  one  tool  set  an  analyst  can  bring  to  the  staff. 
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H  What  is  Hypothesis  Testing? 

*  Field  of  applied  statistics  and 
probability  that  seeks  to  find 
relationships  between  metrics,  data, 
and  variables. 

■  Helpful  to  quantify  risk  or  compare 
statistical  data. 

•  In  this  case  to  find  if  there  is  evidence 
for  targeting  the  General  officer  by 
using  a  standard  probability 
distribution. 


IB  Hypothesis  Testing 


MhJC-I  requested  assistance  in  evaluating  the  likelihood  that  insurgents,  w&re 
targeting  a  Coalition  general  officer  during  his  visits  to  various  FOBs- 

H X  -  BN  (n-2C,  p-U_20)  HpfelO)  <  C 3% 


Bino-pul  i >v=5C.  fy=U.2\  P'CorKui  iy  Mrrj-:,  FwkiIIlhi 


Randomly  seYiertfei}  data 


s  c  7  ■  io  ii  u  ii  i-i  is  it;  u  in  in  vii 

ft  ivTtaMT  at  Vialti  Allsiehiid 


GO  had  made  20  trips.  He  had  received  IDF  on  10  of  the  20.  Ra  ndom? 

Met  with  the  G3-Air  and  Identified  all  flights  in  the  last  month  in  Baghdad. 
Determined  how  many  FOBS  received  IDF  w/m  30  minutes  on  landing  =  20%. 

How  many  independent  flights  out  of  20  should  be  attacked  if  p-0.20? 

How  do  you  explain  0.39^? 
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S  Details 

*  U  n  derlin  g  p  rob  a  bi  lity  d  istri  button  for  th  is  a  n  a  lysis  wa  s 
the  binomial,  or  how  many  "successes"  will  we  have 
from  so  many  trials, 

*  This  analysis  involved  hypothesis  testing  with  HD 
being  the  General’  helo  landing  is  just  another 
random  event  with  probability  Of  .2  that  it  will  trigger 
an  indirect  attack  on  the  FOB, 

*  The  question  then,  is  what  are  the  chances  of 
experiencing  ten  or  more  attacks  in  20  trials  if  each 
“trial"  has  a  probability  of  .2  of  trigging  an  attack, 

*  The  answer  is  less  probability  than  rolling  three  sixes 
with  three  dice:  ,3%.  which  is  how  the  analyst 
describe  this  to  the  general. 


Strengths  and  Weaknesses  of 
Hypothesis  Testing 


*  Does  not  always  discover  “cause  and 
effect”,  but  can  provide  insight  in  relation 
(correlation) . 

*  Requires  a  good  data  base  on  objects  of 
interest 


♦  In  many  cases,  assumes  environment  in  the 
immediate  future  will  be  the  same  as  the 
immediate  past  for  data  consistency. 
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A.2.2  Persistent  Threat  Detection  System 

INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Persistent  Threat  Detection  System 

Faculty 

Name(s) 

Bundeswehr  Planning  Office,  Ottobrunn,  DEU 

Length 

20  -  30  minutes 

Learning 

Objectives 

Understand  the  advantage  of  a  mathematical-based  analysis. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Recognize  similar  applications  where  an  analysis  by  OA  staff  could  support 
planning. 

Content 

Outline 

Case  Study:  PTDS. 

Materials 

None. 

Methods 

100%  Lecture  20%  Class  Exercises. 

%  Break  Outs. 

Exercise(s) 

None. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

Did  you  have  a  similar  case  in  the  past? 

Can  you  think  of  a  current  case  in  your  area  of  responsibility? 
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Persistent  Threat 
Detection  System 

A  Case  Study  In  Risk  Assessment 

Name  Lecturer 
Bundeswehr  Planning  Office 


Overview 


Situation:  A  new  system  -  PTDS  -  had  to  be 
stationed  at  PRT  Kunduz  conflicting  with  the  safety 
area  of  a  drone  --  KZQ  —  take-off  location 

Decision:  Find  a  new  take-off  location  for  KZQ 

Objective:  Locate  a  new  take-off  area  in  compliance 
with  local  conditions 

QA  Contribution:  Risk  assessment  for  several 
possible  KZQ  take-off  locations 


‘Read  points* 


Note: 

PTDS  -  Persistent  Threat  Detection  System  -  is  a  tethered  aerostat-based  system 

KZO  is  the  German  abbreviation  for  “Kleinfluggerat  fur  Zielortung".  KZO  is  a  small 
UAV.  The  next  slide  will  provide  more  details  about  the  PTDS  and  KZO. 
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EH  PTDS  and  KZO 


PTDS  is  a  tethered  .aerostat-based  system 
equipped  with  multi-mission  sensors  to 
provide  tong  endurance  intelligence, 
surveillance,  reconnaissance  and 
communications  in  support  of  coalition 
forces, 


KZQ  is  a  highly  advanced  surveillance, 
reconnaissance  and  large!  localize  lion 
system.  The  payload  used  for  his  specific 
task  is  an  imaging  infrared  sensor  capable 
of  acquiring  highly  accurate  Jala  on 
stationary  and  moving  targets  at  any  time 
of  the  day  or  night 
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™  Problem  Statement  PTDS  -  KZO  (1) 


o  PTDS  was  introduced  in  2011  in  the  area  of  FRT  Kunduz. 

o  The  location  for  the  mooring  system  for  PTDS  was  given  and 
is  therefore  a  fixed  parameter 

o  The  safety  areas  of  the  two  systems  (PTDS  and  KZO)  were 
critically  overlapping  because  the  PTDS  mooring  system  is 
near  the  take-off  location  of  the  KZO. 

o  A  new  take-off  location  for  the  KZO  had  to  be  determined  with 
the  goal  to  minimize  the  overlap  between  the  safety  areas, 

o  Based  on  local  restrictions,  specific  take-off  locations  were 
analyzed  with  regard  to  the  risk  of  overlapping  safety  areas. 
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Here  the  safety  areas  of  the  both  systems  are  visualized.  The  KZO:  depicted  as 
TfO,  takes  off  in  a  north-easterly  direction.  It  has  a  safety  area  with  a  range  of  550 
m.  This  safety  area  is  fixed,  because  it  is  in  relation  to  the  take-off  location  and  not 
the  moving  KZO.  The  PTDS  has  a  safety  area  of  150  m  around  the  airship.  The 
PTDS  and  its  corresponding  safety  area  will  drift  depending  on  speed  and  direction 
of  existing  winds.  The  operational  altitude  of  the  PTDS  is  between  1000  and  1 500 
meters. 
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ski  Analytical  Approach 

1 .  Def i  riitiu  n  of  assurn  ptio  ns 

*  E  g.  height  uf  PTDS  does  not  chpnge  with 
various  weather  Conditions  and  wind  drift 

2.  Extraction  of  relevant  weather  data  for  PTDS 

3.  Estimation  of  the  parameters  for  the  deflection 
formula  of  the  PTDS 

4.  Computation  of  the  PTDS  positions  according  to 
the  deflection  formula  under  consideration  of 
weather  data  from  past  B  months 

5.  Ana  lys  is  of  va  rio  us  take-off  locati  one 

6.  Visualization  of  the  results  (comparison  old  take¬ 
off  location  and  best  possible  new  fake-off 
location) 


The  analytical  approach  was  as  follows.  First  of  all,  assumptions  had  to  be  taker  to  simplify 
the  real  aerostat  drifting  problem.  One  assumption  was  that  the  PTDS  will  not  lose  height,  if  it 
drifts  away  from  the  mooring  system.  After  defining  the  problem,  weather  data  had  to  be 
obtained.  German  artillery  units  collect  wind  speed  and  wind  direction  daily  at  several  heights 
and  times  of  the  day.  The  data  for  an  altitude  of  1000-1500  meters  (operational  height  of  the 
PTDS)  were  extracted  from  existing  text-files  via  Java.  1466  dates  were  extracted  covering  & 
months  of  weather  data  around  PRT  Kunduz.  Then  the  deflection  of  the  PTDS  as  a  function 
of  height  and  wind  speed  had  to  be  determined.  With  this  formula  the  Active  position  of  the 
PTDS  in  the  last  8  months  could  be  calculated.  Several  possible  take-off  locations  could  be 
analyzed  with  regard  to  their  risk  of  overlapping  safety  areas  of  the  two  systems.  Finally,  the 
results  were  visualized  for  easier  understanding.  On  the  next  slides  you  will  get  more  details 
about  the  individual  steps  of  the  analytical  approach. 
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S  Analysis  of  wind  data 


histogram  wind  direction 
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histogram  wind  speed  in  lets 
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Here  the  histograms  of  the  wind  direction  and  wind  speed  are  depicted.  As  you  can 
see,  wind  from  the  northwest  is  quite  frequent.  Therefore,  the  data  suggest  a  new 
take-off  location  northwest  of  the  PTDS  mooring  system.  In  most  cases  the  wind 
speed  was  relatively  low  and  in  99%  of  all  cases  less  than  20  knots.  This  is  an 
important  factor  with  regard  to  the  deflection  formula  depicted  on  the  next  slide. 


P«1  PTDS  Deflection  Model 


This  picture  was  the  only  information  the  analysts  had  with  regard  to  deflection  of  the  PTDS. 
You  can  see  the  deflection  for  3  separate  altitudes  in  feet  and  the  deflection  for  25  and  50 
knots  wind  speed.  However,  data  were  missing  for  a  deflection  at  a  wind  speed  of  5  knots. 
Additionally,  the  problem  was  not  linear.  The  deflection  formula  at  the  bottom  was  specifically 
created  by  the  analysts  for  this  specific  problem  set.  With  this  formula  you  can  calculate  the 
deflection  for  every  height  and  wind  speed. 
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™  Visualization  of  the  original 
take-off  location 


* 


■  *? 


Here  you  can  see  the  visualized  result  for  the  risk  analysis  of  the  original  take-off 
location.  A  coordinate  system  of  boxes  with  a  dimension  of  50  feet  (width/length) 
was  used.  The  safety  area  of  the  KZO  take-off  location  is  shown  in  light  blue.  The 
analysis  showed  how  often  PTDS  would  have  drifted  into  a  box  within  the  last  8 
months.  For  better/easier  understanding  a  heat  map  was  used  based  on  the 
numerical  data.  Green  color  means  that  the  PTDS  drifted  rarely  in  a  specific  box. 
Red  means  that  it  drifted  very  often  in  this  box.  You  can  see  that  the  original  take-off 
location  is  in  close  proximity  of  the  PTDS.  The  data  clearly  show  the  criticality.  As 
you  can  see  with  a  probability  of  22,5%  the  PTDS  would  have  interfered  with  the 
safety  area  of  the  KZO.  This  high  risk  is  not  acceptable. 
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™  Visualisation  for  best 
possible  take-off  location 


Here  you  see  in  comparison  the  results  of  the  best  possible  (with  regard  to  local 
restrictions)  take-off  location.  The  probability  that  the  PTDS  will  drift  into  the  safety 
area  of  the  KZO  reduced  to  2,4%  from  over  20%,  as  seen  on  the  previous  slide. 
Additional,  in  no  case  did  the  PTDS  drift  into  the  take-off  con-id  or  of  the  KZO.  Based 
on  this  risk  assessment,  the  commander's  decision  was  to  replace  the  original  take¬ 
off  location  with  this  one. 


What  is  Risk  Assessment? 


•  Risk  assessment  is  the  determination  of 
quantitative  and  /  or  qualitative  value  of  risk 
related  to  a  concrete  situation  and  a  recognized 
threat  (also  called  hazard). 

•  In  our  case,  to  determine  the  risk  that  the  PTDS 
will  drift  into  the  safety  area  of  the  KZO. 

•  For  the  analysis  mathematic  and  statistic 
elements  were  used. 
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ra  Strengths  and  Weaknesses 
of  this  Example 

*  Here  the  analysis  was  based  on  historic  data  and 
does  not  predict  future  wind. 

•  To  cope  with  this  weakness,  critical  combinations 
wind  speed  (i.e.  >  20  kts)  and  wind  direction  (i.e. 
030°)  were  used. 

■  For  better  comprehension  of  the  results, 

visualization  was  used. 

•  Fast  results:  Analysis  took  1  week. 

*  Operational  art  was  supported  by  science. 
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Major  Points 


*  To  best  utilize  your  operational  analyst  to  aid  in 
decision  making,  they  need  access  to  you,  the 
decision  maker. 


■  The  analyst  brings  a  variety  of  tools  (optimization, 
simulation,  statistics,  and  assessment  skills)  to 
help  you  with  evidence  based  decision  making. 
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Impressum 


•  Created  by:  Bundeswehr  Planning  Office 

■  References:  TBD 

•  Materials:  TBD 

»  Lecturer:  LTC  Burgert 
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A.2.3  Intra-Theatre  Airlift 

INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Using  Operational  Analysis  to  Improve  Intra-Theater  Lift:  A  Case  Study  in 
Optimization 

Level 

Operational 

Faculty 

Name(s) 

Robert  Dell,  Naval  Postgraduate  School,  Monterey,  California,  USA 

Length 

20  -  30  Minute  General  Briefing;  40  -  60  minutes  Specialist  Briefing 

Learning 

Objectives 

Understand  how  optimization  can  contribute  to  efficient  scheduling  in 
operations. 

Understand  the  concept  of  optimization  (exact  and  heuristic). 

Appreciate  its  contribution  in  a  complex  planning  environment. 

Understand  through  case  example  the  use  of  optimization  in  logistics 
scheduling. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Appreciate  the  uses  of  optimization  in  logistics  scheduling  and  understand  its 
strengths  and  limitations. 

Content 

Outline 

Case  Study:  Intra-theater  airlift. 

Optimization  and  Heuristic  Defined. 

Materials 

List  Required  Pre-Readings: 

None. 

Methods 

100%  Lecture  0%  Class  Exercises. 

%  Break  Outs. 

Exercise(s) 

Seminar  participants  will  be  asked  to  evaluate  charts  and  outputs  from  each 
case  study. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

What  are  your  experiences  with  manual  scheduling? 

How  might  optimization  improve  that  process? 
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Improving  Intra-Theater 
Airlift  in  Iraq  to  avoid  IEDs 


A  Case  Study  In  Optimization 

*  zrig/zc'i  3  *  i 


This  presentation  shows  a  case  study  in  the  use  of  operational  analysis  to  improve  intra- 
theater  air  lift  scheduling  during  the  Iraq  conflict. 


Overview 


Situation:  Iraq  2005,  IED  Threat  to  Truck  Convoys 

Decision:  Use  airlift  to  lessen  exposure  of  ground 
troops 

Objective:  Move  as  much  as  possible  using  airlift 

OA  Contribution:  Optimized  use  of  aircraft 
credited  with  saving  lives 


Outcome:  More  aircraft  available  for  logistics  lift 


In  2005  the  IED  threat  was  growing  to  truck  conveys  carrying  supplies  to  forward  operating 
bases  in  Iraq.  To  avoid  ground  troop  exposure  to  IEDsr  It  was  the  desire  to  move  as  many 
supplies  as  possible  via  air  lift.  As  we  will  see,,  using  operational  analysis  techniques  of 
optimization,  an  improved  air  lift  schedule  was  created  that  actually  freed  aircraft  for  more 
scheduling. 
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A  7-day  Iraq  demand 

Move:  15,677  passengers  and  2,027  pallets 

With:  17  airfields 

Up  to  30  aircraft  stationed  at  five  bases: 
23  C-130E,  5  C-17, 2 IL-76 


This  is  a  snap  shot  of  the  problem.  This  is  a  typical  7-day  demand  schedule  showing  the 
number  of  PAX  and  pallets  requested  for  movement  between  specific  origins  and 
destinations.  This  week  we  can  see  a  request  to  move  a  total  of  over  15rQOO  passengers 
and  over  2000  pallets.  The  theater  air  assets  to  accomplish  this  17  destination  airfields, 
and  thirty  lift  aircraft  stationed  at  five  bases. 


How  to  best  move  this  demand? 

•  Ask  for  more  airlift 

•  Ask  for  larger  capacity  aircraft 

•  Improve  airlift  efficiency  by; 

o  Checking  air  logistics  processes  such  as 
loading  and  unloading,  refueling,  and 
crew  rotation  for  hindrances  or  slack, 

o  Improve  scheduling  of  current  processes. 

_ r* 


Alternatives  to  lift  this  demand  are  listed  on  the  slide.  Although  all  were  considered,  this 
study  focuses  on  improving  the  scheduling  of  the  air  lift  processes. 
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OA  Contribution 


There  are  272,02a  total  possible  routes:  too 
many  to  manually  explore  for  "best" 
scheduling  solution 

Using  an  operational  analysis  technique 
called  optimization  with  COTS  software: 

>  Only  10  aircraft  are  needed  to  move 
almost  all  demand  (99.2%)  with  smart 
scheduling. 

>  Frees  20  aircraft  for  emergent  demand 


With  all  possible  aircraft  and  air  base  assignments,  manually  exploring  ail  alternatives  to 
find  the  best  schedule  to  move  demand  in  near  impossible.  Using  optimization,  however, 
the  analysts  produced  a  scheduling  tool  that  resulted  in  lifting  over  99%  of  the  demand 
with  on  only  10  aircraft.  This  freed  up  20  aircraft  for  addition  lift  requirements. 


ygujl 

[5221 

What  is  Optimization? 

•  Field  of  applied  math  that  seeks  to  find 
the  most  cost  effective  or  best 
performing  alternative  under  given 
resource  constraints  by  leveraging 
desired  factors  or  minimizing  undesired 
ones. 

•  In  the  airlift  case,  to  find  best  use  of 
airlift  capacity  through  smart 
scheduling. 


The  field  of  optimization  can  be  applied  in  many  ways:  from  scheduling  to  assignment 
problems.  It  usually  requires  an  analyst  familiar  with  the  tool,  and  an  operator  that 
understands  the  system  to  be  optimized. 
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Strengths  and  Weaknesses  of 


Optimization 

*  May  be  seen  as  an  “If”,  'Then’1  statement: 
“if"  this  data,  "then"  this  is  the  best  course  of 
action:  need  good  data. 

*  Optimizing  versus  Satisficing:  Ideal  schedule 
versus  reality  or  the  plan  meets  execution, 

*  Great  asset  for  quick  turn  around  answers  to 
resource  allocation 


A  very  valuable  use  of  optimization  is  applying  it  to  "What  if"  problems.  If  this  assumption 
or  data  set  is  true,  what  is  the  best  we  can  do  compared  to  if  another  assumption  or  data 
set  is  true.  Another  case  study  example  of  this  is  scheduling  combat  logistics  force  ships  in 
the  U.S.  Fifth  Fleet  ADR  where  the  commander  needed  to  understand  the  impact  of  losing 
an  oil  replenishment  ship. 


Major  Points 

*  To  best  utilize  your  operational  analyst 
to  aid  in  decision  making,  they  need 
access  to  you,  the  decision  maker. 

■  The  analyst  brings  a  variety  of  tools 
(optimization,  simulation,  statistics,  and 
assessment  skills)  to  help  you  with 
evidence  based  decision  making. 
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More  Detail  Follows 
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The  following  slides  may  be  used  for  more  detail  to  the  problem.  They  show  flight  routes, 
bases,  air  assets  and  capabilities  and  provide  the  formulation  for  ATEM,  the  air  scheduling 
model. 


CE5S 

[J2SI 

Intra-theater  Airlift  Scheduling 

*  ATEM  (Air  Tasking  and  Efficiency  Model] 
developed  to  assist  with  Intra-theater  airlift  in 
Iraq  and  Afghanistan 

*  Use  started  in  2006  for  both 

o  Frequency  channel  routes  (strategic] 

*  published  schedule  using  a  subset  of 
aircraft 

c  Requirements  channel  routes 
(operational] 

*  planned  daily 
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Schedule  What? 

*  Intra-theater  airlift  of  two  basic  commodities: 
o  Pallets  (PALS) 

o  Passengers  (PAX] 

*  ...  between  17  airfields ... 

■  ...  using  three  basic  airframes: 
oC-17  ($20GM  each} 
o  C-13DE  ($4QM  each].  Jl,  J2 
o  IL-76  (contract  aircraft) 

*  Goal:  Move  as  much  materiel  as  possible 


li  Schedule  Where? 
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Schedule  How? 


*  Combination  ot  two  basic  ideas: 

o  Constrained  enumeration  of  all  feasible  routes 
(a  priori  column  generation) 

o  Integer  linear  programming  (ILP)  to  select  “best" 
routes 

*  GAMS/CP  LEX 

*  Greedy,  constructive  heuristic 


Schedule  What? 


*  Intra-theater 

a  C-17  Globemaster  ($200M  each) 
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C-17  Configurations 

conf 

pax 

pals 

cl7c 

0 

18 

cl7cp 

54 

11 

cl7p 

189 

4 

Must  also  sefect  a  configuration  for  each  aircraft  on  each  route 

■f 
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nrm 

Schedule  What? 
•  Intra-theater 

cC-17  ($200M  each) 
o C-l 30E  ($40M  each),  Jl,  J2 


■f  ■ 


ANNEX -42 


STO-TR-SAS-098 


ANNEX  TO  SAS-098 


ES 

C-130E  Configurations 
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C-130J  Configurations 
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Schedule  What? 

•Intra-theater 

oC-17  ($200M  each) 
oC-1 30E  ($40M  each),  Jl,  J2 
olL-76  (contract  aircraft) 
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IL-76  Configuration 

conf  pax  pals 

0  9 


Schedule  What? 


•  Intra-theater 

oC-17  ($200M  each} 
oC-130E  ($40M  each),  Jl,  J2 
o  IL-76 

•  itinerant  C-5 

■f  ■ 
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Move  What? 

Line 
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™  Demand  Lines 
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Routes 
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How  to  route? 

*  Aircraft  must  start  and  end  at  same  airport 

*  Time  between  airports  by  aircraft  type 

*  Ground  time  by  aircraft  type  and  airport 

*  Refueling  time  by  aircraft  type  and  airport 

*  Maximum  landings  in  the  box  before 
recrewing 

*  Demand  line  "throughput”  allowed 
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Operating  Limits 


017 

C-130E 

Mx 

route  hrs 

10.0 

8.0 

Mx 

hrs  bef  refuel 

10,0 

5,0 

Mn 

hrs  bef  refuel 

10.0 

3.0 

Mx 

lands  in  "box" 

5 

20 

Mx 

re crews 

1 

0 

Aircraft  and  Cargo 


up  to  35  aircraft 

o  up  to  five  types 
q  multiple 
configurations 

17  airfields 

o  five  home 
airfields 

cargo  types 

o  passengers 
o  pallets 


7/1012013 
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liBHI 

E3E1 

How  many  routes? 

•  For  a  fleet  consisting  ot; 

Thirly  pic rw:  . 

c,  ...  of  ihree  bosk:  plans  Types . . . 

-  C-l  7,  C-130E.IL-7.fi 
o  .  basetfai  Jiv&slortineoi^kJs, 

*  We  get  o  few  hundred  thousand  routes  to  consider 

*  Use  an  optimization  model  to  select  the  best  set 

•  A  developed  heuristic  can  select  a  good  set.  The 
heuristic  has  no  solution  guarantee  (don't  know  if  you 
can  do  better)  but  it  is  quicker  and  doesn't  require 
special  software 


■p  ■ 


g  1-day  Afghanistan  example 

40  demand  lines:  404  passengers,  844  pallets 
14  airfields 

14  aircraft  at  two  home  airfields:  6  C-130E,  8  C-17 


Demand  Lucs 
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1-day  Afghanistan  example 

1,916  total  routes 

3  secs  to  find  optimal  solution  (ILP) 
less  time  using  the  heuristic 

both  heuristic  and  ILP  solutions  move 

-  155  of  404  passengers 

-  398  of  844  pallets 

-  only  TLP  lets  you  know  that  you  can’t  do 
better 


HI  1-day  Iraq  scenario 

82  demand  lines:  505  passengers,  645  pallets 
17  airfields 

29  aircraft  at  five  home  airfields: 

22  C-130E,  5  C-17, 2  IL-76 


Demand  for  l-n  rge  one-day  seen*  rfo  ■  PALS 

□  FAX 
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™  1-day  Iraq  scenario 

207,121  total  routes 

2  minutes  to  generate  all  routes 
40  minutes  for  ILP  (0.01  gap) 

3  minutes  for  the  heuristic 
AT  EM  ILP  moves 

-  445  of  5(55  passengers  (88%) 

-614  of  645  pallets  (95%) 

ATEM  heuristic  moves 

-  456  of  505  passengers  (90%) 

-  588  of  645  pallets  (91%) 


■f 


* 


7-day  Iraq  scenario 

85  demand  lines:  15,677  passengers,  2,027  pallets 
17  airfields 

30  aircraft  at  five  home  bases: 

23  C-130E,  5  C-17,  2 IL-76 


2KB 

^  1KB 

■a 

a  ig» 
sx 

Q 


Demand  for  seven-day  scenario 


Jln^.n-11  nl  ifl 

Ia  hi  ■f  d  Linux 
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7-day  Iraq  scenario 


272,024  total  routes 

2  minutes  to  generate  all  routes 

3  minutes  for  ILP  (0.01  gap) 

9  minutes  for  the  heuristic 

both  heuristic  and  ILP  solutions  move 
nearly  all  the  demand 

Mr 


Seven-day  scenario 


30  aircraft? 


What  if  we  si  art  removing  some? 
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*  ,■  ■  iu- jVI  ^  h3 

§  ATEM 

(Air  Tasking  and  Efficiency  Model) 

Used  extensively  for 

oDaily  planning 
oChannel  design 
oHub  and  spoke  evaluation 
oFleet  mix 
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Bl  Indices  and  index  sets 

at  A 

m  of  »lrfleld  AFOs  [-  10] 

ceC 

scl  of  cargo  ijjhjs  ipiLss^njjrsrj  und  turgu  pul  Ids)  [- 1] 

P<lF 

sel  ofplaTte  lypra  [—  _s] 

P*PS 

set  of  places  of  type  p  bated  as  airfield  a 

set  of  canfiguraliofis  for  plane  type  /?  |  -  5] 
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set  of  flight  kp  or  sapnenMAPOE*  APOD  pairs)  [-  I  Ou]J 
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airfield  lt  is  cktLelo  (APOE1  of  leg  i 
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Data 

amount  of  cargo  c  on  leg  1  dial  can  be  airlifted  by  route  r 

priority  of  cargo  c  on  leg  / 

demand  units  of  demand  li  ne  /  for  cargo  c 

ccrjP@e 

capacity  of  configuration  g  for  cargo  lypet 

maximum  number  ^Handings  in  APO  a 

number  of  plane  type  p  starting  from  APO  airfield  a 

maximum  number  of  routes  to  select  for  each  plane  type  p 
in  each  configuration  g  starling  from  each  airfield  a 

■f 

■ 
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Variables 


SELECT 'r  integer  number  of  times  plane  type  />(r) 

is  flown  on  route  r 

EXTRA fc  integer  capacity  of  selected  routes  to  take 

extra  cargo  cs  leg  f  demand  if  available 
LOAD  >c  integer  cargo  units  airlifted  by  plane  type  p(r) 
flying  route  r  from  AIJOE  airfield  s  to  APOD 
airfield  s'  of  cargo  type  c 


IfeHISI 

™  Without  Throughput 

Maximize 

I  q>k  ■  SEL  ECTr  -  £  cpk  EXTR  Ah 

Scr  ic 

Subject  to: 

V  SELECT^  <  planes pQ 

Vu,p€/;f 

V  carry j€KSELECTr  <  dem[t.  +  EXTRAit. 

r 

V/,c 

SEL  ECT,  <  j 'andinga 

r. 

Vtr 

■f 

■ 
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™  With  Throughput 

Subject  to: 

X  SELECT, :  <  planes  ^ 

reftp, , 

X  LOAD„tc  <  demk 

FP 

,f  )^r(  f] 


Va,pePa 

V7,c 


y  £0*4  Drsfc  £  ^  5ELFC7],  Vr ,  s' "  e  5,. ,  c 


7;  .SEifCr,  <  landing^  Via 


With  Throughput 


Maximize 


Z  CPciL0ADrss‘c 

r,x^Sr  y^sfri. 

/|cr(  r  >s  )=a(  l )  Act  { r  ,s  r)=t?r(  /), 
a|  $r>$+c 
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A.2.4  Peace  Support  and  Game  Theory 

INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Decision-Making  Process  in  Peace  Support  Operations 

Faculty 

Name(s) 

Asst.Prof.Dr.  Col.  Hakan  Soner  APLAK,  Turkish  Military  Academy,  Ankara, 

Turkey 

Length 

75  -  90  minutes 

Learning 

Objectives 

Provide  an  approach  for  situation  awareness. 

Provide  a  hybrid  operational  analysis  approach  for  decision-making  in 
uncertainty. 

Provide  an  analytic  method  for  evaluation  of  strategies. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Analyze  situation. 

Utilize  the  analysts  to  aid  in  operations  and  decisions  in  planning. 

Content 

Outline 

Information  about  situation. 

Proposed  approach  for  decision-making. 

Contributions  of  OA  to  decisions. 

More  Detailed  information  about  application. 

Materials 

List  Required  Pre-Readings: 

Yager,  R.R.,  “Multiple  Objective  Decision-Making  Using  Fuzzy  Sets”,  International 
Journal  of  Man-Machine  Studies,  1977. 

Chen  S.J.  and  Hwang  C.L.,  “Fuzzy  Multiple  Attribute  Decision  Making:  Methods 
and  Applications”,  Springer,  New  York,  1992. 

Ross,  T.,  “Fuzzy  Logic  with  Engineering  Applications”,  2nd  Ed.  John  Willey  & 

Sons  Ltd.,  West  Sussex,  England,  2004. 

Godet,  M.,  “Scenarios  and  Strategic  Management”,  London  Butterworth,  1987. 

Aplak,  H.S.,  “Fuzzy  Logic  Based  Game  Theory  Applications  in  Decision  Making 
Process”,  Ph.D.  Thesis,  Gazi  University,  Institute  of  Science  and  Technology, 

2010. 

Aplak,  H.S.  and  Turkbey  O.,  “Fuzzy  Logic  Based  Game  Theory  Applications  in 
Multi-Criteria  Decision  Making  Process”,  Journal  of  Intelligent  and  Fuzzy 

Systems,  2013. 

Aplak,  H.S.  and  Turkbey  O.,  “The  Application  of  Two  Person  Non-Constant  Sum 
Games  in  Multi-Objective  Decision  Process”,  The  Journal  of  the  Faculty  of 
Engineering  and  Architecture  of  Gazi  University,  2013. 
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Methods 

100%  Lecture  0%  Class  Exercises. 

%  Break  Outs. 

Exercise(s) 

Participants  will  be  ask  to  discuss  relevancy  of  approaches  to  their  past  decision 
challenges. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

What  are  your  experiences  about  peace  keeping  operations? 

How  might  game  decision  payoff  matrix  be  obtained? 

How  can  analytical  methods  be  used  in  DM  process  in  uncertainty? 

Peace  Support  Operation  Planning 
(Game  Theory  Application  in 
Decision  Making  Process) 


A  Case  Study  in  Game  Theory 

11 J 07.2013 


Peace  Support  Operation  Planning  (Game  Theory  Application  in 
Decision  Making  Process) 

A  Case  Study  in  Game  Theory 

Asst.Prof.DrXol.  Hakan  Soner  APLAK 
Turkish  Military  Academy 
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Overview 


Situ  alien:  This  is  a  generic  scenario  obout  an  internatiQinal 
disagreement  which  was  created  by  the  inspiration  of  recent  past 
events  and  NATO  exercises.  Countries  (Alpha  and  Beta)  have 
problems  in  politics,  economics  ond  social  matters  since  history. 


Decision:  Regarding  to  this  situation,  evaluate  silualion  in  decision 
making  process  ond  analyze  strategies  according  to  delermlned 
objectives  and  criteria. 

Objective.  Determining  optimal  decision  according  lo  situation. 


OA  Contribution:  Presenting  a  hybrid  decision  making  approach 
vvhich  uses  artificial  intelligence  techniques  (game  theory  and  fuzzy 
set  theory)  in  peace  keeping  operations  planning. 

Result  or  outcome:  Obtaining  an  optimal  strategy  for  peace  support 
Forces  regarding  to  objec lives. 

1 1 JD7J30I  a  2 


Situation:  This  is  a  generic  scenario  about  an  international  disagreement  which  was  created  by  the  inspiration 
of  recent  past  events  and  NATO  exercises.  Countries  (Alpha  and  Beta}  have  problems  in  politics,  economics 
and  social  matters  since  history. 

pecj|jjpp^  t ^ n^ij ffiVt /nJ 3  * '  °  n  j  evaluate  situation  In  decision  making  process  and  analyse  strategies 
Objective* Determining  optimal  decision  according  to  situation. 

OA  Contribution:  Presenting  a  hybrid  decision  making  approach  which  uses  artificial  intelligence  techniques 
I  ga  me  t  heor^  a  nd  fuzzy  I  ogic  system )  in  peace  kee  pi  n  g  o  perations  pla  n  n  in  g . 

Result  or  outcome:  Obtaining;  an  optimal  strategy  for  peace  support  forces  regarding  to  objectives. 


IS  Situation 


•  This  is  a  generic  scenario 
about  an  international 
disagreement. 

*  It  includes  a  peace  support 
operation  planning. 
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This  is  a  generic  scenario  about  an  international  disagreement. 
It  includes  a  peace  support  operation  planning. 


ANNEX  -  60 


STO-TR-SAS-098 


ANNEX  TO  SAS-098 


Situation 


BETA 


llJOUAU 


•  Countries  (Alpho  ond  Beta) 
are  in  disagreements  in 
politics,  economics  and  social 
matters  since  history. 

■Beta  claims  that  Buffer  Zone 
(BZ)  area  which  has  plenty  of 
natural  sources,  belongs  to  it. 

•  Alpha  call  for  an 
international  aid  against 
Beta's  demand.  United 
Nations  send  and  located 
international  peace  support 
force  to  control  the  area. 


The  formed  case  is  a  generic  scenario  and  about  an  international  disagreement  between 
two  countries.  It  can  be  summarized  as  follows; 

■  Countries  (Alpha  and  Beta)  have  problems  in  politics,  economics  and  social  matters  since 
history. 

4  Especially,  Beta  claims  that  Buffer  Zone  (BZ)  area  which  has  plenty  of  natural  sources, 
belongs  to  it. 

■  For  this  reason.  Alpha  call  for  an  international  aid  against  Beta's  demand.  United  Nations 
send  and  located!  international  peace  support  force  to  control  the  area,  especially  buffer 
zone. 


Approach 


*  The  situation  is  analyzed  in  two  person  non-constant 
sum  game  theory  perspective  and  a  hybrid  decision 
making  approach  is  used  for  Decision  Making  Process 
[DMP], 


*  Qualitative  and  quantitative  methods  ore  used  to 
merge  intuitions,  experiences,  preferences  and 
knowledge  of  decision  makers  [DMs)„ 
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*  The  situation  is  analyzed  in  two  person  non-constant  sum  game  theory  perspective  and  a 
hybrid  decision  making  approach  is  used  for  Decision  Making  Process  (DMP). 

*  Qualitative  and  quantitative  methods  are  used  to  merge  intuitionsr  experiences, 
preferences  and  knowledge  of  decision  makers  (DMs). 
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Steps  of  Decision  Making  Approach 

*  Analyzing  the  situation. 

*  Defining  objectives,  strategies  and  criteria, 

■  Determining  linguistic  variables. 

*  Calculating  objectives’  importance. 

■  Analyzing  strategies  according  to  objectives, 

■  Calculating  criteria's  importance. 

*  Analyzing  strategies  according  1o  evaluation  criteria. 

■  Forming  decision  payoff  matrix. 

-  Finding  equilibrium  point  and  optimal  strategies, 

11  J07.2&I3  4 


Steps  of  proposed  decision  making  approach  are^ 

*  Analyzing  the  situation. 

*  Defining  objectives,  strategies  and  criteria. 

*  Determining  linguistic  variables. 

*  Calculating  objective  importance. 

*  Analyzing  strategies  according  to  objectives. 

*  Calculating  criteria's  importance. 

*  Analyzing  strategies  according  to  evaluation  criteria. 

*  Forming  decision  payoff  matrix. 

*  Finding  equilibrium  point  and  optimal  strategies. 
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S  Methods 

*  Research  techniques  (questionnaire,  interview). 

*  Statistical  mod  to  calculate  the  value  of  linguistic 
variables. 

*  Fuzzy  set  theory  and  fuzzy  linguistic  variables  to 
calculate  objectives  and  criteria's  importance's . 

■  Multi-objective  decision  making  method  to  calculate 
strategies  performances  according  to  objectives. 

■  Fuzzy  TO  PS  IS  method  to  calculate  strategies 

performances  according  to  criteria. 

*  Game  theory  to  frame  whole  process  and  to  obtain  an 
optimal  solution. 

iijojjfiia  7 


Methods  which  are  used  in  this  approach; 

*  Research  techniques  (questionnaire,  interview). 

*  Statistical  mod  to  calculate  the  value  of  linguistic  variables. 

*  Fuzzy  set  theory  and  fuzzy  linguistic  variables  to  calculate  objectives  and  criteria's 
importance's. 

*  M  ulti -objective  decision  making  method  to  calculate  strategies  performances  according 
to  objectives. 

*  Fuzzy  TO  PS  IS  method  to  calculate  strategies  performances  according  to  criteria. 

*  Game  Theory  to  frame  whole  process  and  to  obtain  an  optimal  solution. 
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Objectives  for  Players 

*  After  analysis  of  situation,  four  objectives  are 
identified  for  each  player.  Objectives  are  defined 
for  each  player  in  fhe  same  perspective  but 
opposite  direction. 

*  While  Alpha  is  trying  to  maximize  the  satisfaction 
level  of  objectives,  Beta  is  trying  to  minimize. 


*  Decision  m  a  king  group  (DMG)  is  consisted!  by  experts  who  have  similarity  in  their 
education  level  and  had  experiences  about  case. 

*  Situation  is  analyzed  by  DMG,  then  objectives  and  strategies  of  players  are  identified. 

*  After  analysis  of  situation,  four  objectives  are  identified  for  each  player. 
Objectives  are  defined  for  each  player  in  the  same  perspective  but 
opposite  direction. 

*  While  Alpha  is  trying  to  maximize  the  satisfaction  level  of  objectives.  Beta  is 
trying  to  minimize. 
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Objectives  Of  Alpha  Player 

*  Protecting  Buffer  Zone 

*  Protecting  whole  responsibility  area 

*  Controlling  in/out  public  opinion 

*  Peacekeeping 

II.07.2&I3  9 


Objectives  of  Alpha  Player: 


*  Protecting  Buffer  Zone 


*  Protecting  whole  responsibility  area 


*  Controlling  in/out  public  opinion 


*  Peace  keeping 
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Strategies  for  Players 

*  Main  and  rear  area  control 

*  Sector  control 

*  Area  control  as  whole 

*  Strong  area  control  as  whole 

*  Area  control  with  local  forces 

Beta; 

*  Attack  by  itself 

*  Corporate  attack 

*  Passive  attitude 

*  Ownership  of  conflict  zone  [BZ] 

iucv.ma  to 


In  th  is  step,  five  strategies  for  Alpha  (AS),  four  strategies  for  Beta  player  (BS)  are  listed. 

Alpha: 

*  Main  and  rear  area  control 

*  Sector  control 

*  Area  control  as  whole 

*  Strong  area  control  as  whole 

*  Area  control  with  local  forces 
Beta: 

*  Attack  by  itself 

*  Corporate  attack 

*  Passive  attitude 

*  Ownership  of  conflict  zone  (BZ) 
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EMwffl  Criteria  for  Evaluation 

♦  Criteria  -1 :  Management 

•  Criteria  -2:  Protection 

•  Criteria  -3:  Mobility 

*  Criteria  -4:  Logistic 

*  Criteria  -5;  Flexibility 

*  Criteria  -6\  Simplicity 

njD7j»ia 


Thenr  key  criteria  are  determined  by  using  Godefs  Scenario  Planning  sub  method 
"structure  analysis" 

*  Criteria -1:  Management 

*  Criteria  -2:  Protection 

*  Criteria  -3:  Mobility 

*  Criteria  -4:  Logistic 

*  Criteria -5:  Flexibility 

*  Criteria  -  6:  Simplicity 


Linguistic  Variables  for  Evaluation 


Impailoncg_ fvtiY  Humbtr 


Exlreme  Imporionce  (E) 

very  Good  (vG) 

(0.S1 

1.00 

L00] 

Very  High  |VHJ 

Good  (GJ 

([>.69 

D.fll 

0,97) 

High 

Meeftjrn  Good  ]MG) 

(0,54 

0,69 

0,31] 

Medium  [M) 

Medium  |M) 

([>,37 

0.54 

0,69\ 

Lqw  (L| 

Medium  Poor  (MPj 

([>,24 

0,37 

0,54] 

Very  Low  fVLf 

Poor  (P| 

(0,11 

0,24 

0,37] 

No  Importance  [N> 

Very  Poor  (VP] 

(0,00 

0.00 

0,24] 

iU07.3&ia 


12 


*  A  questionnaire  is  applied  to  DMG  to  determine  linguistic  variable's  scale. 

*  Triangular  Fuzzy  N  umbers  (TFN)  are  used  for  describing  the  values  of  linguistic  variables 
which  are  computationally  efficient  for  DMP. 

*  After  getting  subjective  assessments  from  DMG,  these  are  aggregated  by  using  statistical 
mod. 
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B  Payoff  Matrix  Regarding  to  Objectives 


Ptaver-2  I'Eeial1 

ts. 

BS, 

SS, 

P  by01’® 

0.441 

Gl5?4 

0,403 

0,54? 

■D.5E3 

0,370 

0,542 

0.53B 

*5, 

0.427 

0.562 

0.470 

0.570 

0503 

3.225 

0.423 

0.623 

*5.3 

0.X3 

0.414 

0.30? 

04*4 

0.+37 

0£CQ 

0,654 

a$h 

D.4TD 

QSDO 

0.347 

0 .563 

0. 704 

0.35? 

QA&1 

0.500 

a5s 

0.132 

0.442 

0.144 

0.3fi6 

0510 

&.5?2 

0,427 

0.3S3 
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*  Multi-objective  decision  making  (Yager,  1977)  method  is  applied  in  this  phase  of 
methodology. 

*  First,  the  objectives  are  evaluated  for  each  player  and  importance's  are  calculated  by 
using  linguistic  variables  by  DM  G. 

*  Second,  the  strategies  of  each  player  are  evaluated  according  to  objectives.  This  step  is 
implemented  in  game  theory  perspective  and  strategies  are  compared  with  other  player's 
strategies  reciprocally. 

*  Finally,  importance  and  performance  coefficients  are  combined  and  a  final  coefficient  is 
calculated  which  indicates  the  performance  of  each  strategy  for  the  satisfaction  of 
objectives. 
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Payoff  Matrix  Regarding  to  Criteria 


Ptiyer-2  ISeial 
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0,507 
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0.610 
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0.654 

0.^23 

0.499 

0,451 

0.461 

O.S62 

AS, 

0.352 

0,47? 

0,37? 
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0.50S 
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0,407 

0640 

AS* 

0.4&5 
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0,460 

0.617 

0.43V 

Q.fflO 

A5S 

0.330 
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0.141 

0654 

0,520 

0631 

0.513 
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Fuzzy  TO  PS  IS  (Chen  and  Hwang,  1992)  method  is  applied  in  this  phase  of  methodology.  This 
step  is  implemented  in  game  theory  perspective  and  strategies  are  compared  with  opposite 
player  strategies  reciprocally. 

*  Key  criteria  and  linguistic  variables  are  used  for  this  comparison. 

*  This  is  an  evaluation  respect  to  each  strategies  of  opposite  player  and  closeness 
coefficients  are  calculated. 

*  These  coefficients  are  used  to  present  the  power  of  each  strategy  respect  to  opposite 
players'  one  and  a  payoff  matrix  is  formed. 

*  This  payoff  matrix  shows  the  performance  and  satisfaction  degree  of  strategies  according 
to  key  criteria. 
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Decision  Payoff  Matrix 


Pkw-2  [E*  lai 
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0274 
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0.294 
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0.264 

D.24Q 

0.24-1 

■0.205 

HI  01 

0,1 91 
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0.P07 

0.2*4 
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0.263 

0306 

0.213 

02*4 

0.355 

AS, 

0.1-54 

0.232 

Q.M* 

0270 

0364 

0190 

0.250 

a$5 
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0224 

0.053 
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0.2B2 
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0.196 
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Finally,  Decision  Payoff  matrix  is  found. 

*  The  performance  coefficients  which  are  calculated  for  objectives  and  criteria,  are 
combined  with  scalar  multiplication  and  final  coefficients  are  found  for  each  case  (Decision 
payoff  matrix). 

*  As  a  final  phase  of  solution,  the  equilibrium  point  is  found  for  game.  The  optimal 
strategies  are  determined  ( AS1&  BS4). 


Follow-on  actions  supported  by  OA 

■  Optimal  Strategies; 

--  Alpha;  Main  and  rear  area  control  (AS,) 

-  Beta;  Ownership  of  conflict  zone  (BS,) 
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■Optimal  Strategies; 

-  Alpha;  Main  and  rear  area  control  (AS^ 
--  Beta;  Ownership  of  conflict  zone  (BS^) 
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3  OA  Contribution 

*  Analyzing  Situation 

o  What  are  the  objectives,  criteria  and  their 
importance  for  evaluating  strategies. 

o  Which  method  can  be  used  for  evaluation  process. 

•  Evaluating  the  strategies 

o  How  are  strategies  compared? 
o  How  can  decision  payoff  matrix  be  found? 
o  How  can  optimal  strategies  be  found? 
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OA  Contributions  can  be  listed  as; 

Analyzing  Situation 

What  are  the  objectives,  criteria  and  their 
importance  for  evaluating  strategies. 

Which  method  can  be  used  for  evaluation  process. 

Evaluating  the  strategies 

How  are  strategies  compared? 

How  can  decision  payoff  matrix  be  found? 

How  can  optimal  strategies  be  found? 
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Summary 


*  Decision-making  is  the  process  of  making  choices 
among  alternatives  according  to  environmental 
effects. 

*  Environmental  effects  are  complex  and  imprecise.  In 
competition  environment,  there  are  actors  who 
behaves  strategically  to  realize  their  objectives. 

*  Military  Decision  making  is  also  complex  and  includes 
uncertainty. 
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if  it  is  summarized; 

*  Decision-making  is  the  process  of  making  choices  among  alternatives  according 
to  environmental  effects. 

*  Environmental  effects  are  complex  and  imprecise.  In  competition  environment, 
there  are  actors  who  behaves  strategically  to  realize  their  objectives. 

*  Military  Decision  making  is  also  complex  and  includes  uncertainty. 


Uli  Summary 


*  Without  approach  provided  by  OA  Ihere  might  have 
been  no  thorough  analysis  of  the  decision  making  in 
imprecise  competition  environment. 

*  To  best  utilize  your  OAs  to  aid  in  decision  making,  they 
need  access  to  you.  the  decision  maker. 

*  The  OA  brings  a  variety  of  tools  like  optimization, 
simulation,  statistics  (e.g.  Game  theory),  and  assessment 
skills  (fuzzy  logic)  to  help  you  with  evidence  based 
decision  making. 
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*  Without  approach  provided  by  OA  there  might  have  been  no  thorough  analysis  of  the 
decision  making  in  imprecise  competition  environment. 

*  To  best  utilize  your  OAs  to  aid  in  decision  making,  they  need  access  to  you,  the  decision 
maker. 

*  The  OA  brings  a  variety  of  tools  like  optimization,  simulation,  statistics  (e.g.  Game 
theory),  and  assessment  skills  (fuzzy  logic)  to  help  you  with  evidence  based  decision 
making. 
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What  is  Game  Theory?  [Koiier  et  ai.,1994.) 


GAME 

THEORY 


The  game  theory  allows  mathematical 
solutions  of  conflicting  situations.  If  is 
interested  in  situations  in  which  a  DiVhs 
behavior  affects  not  only  their  own 
profits  or  losses,  but  also  those  of 
opposing  DMs. 


*  A  strategic  gome  consists  of  set  of  players,  strategies 
for  each  player  and  payoff  for  each  strafegy 
combination,  A  solution  to  a  game  h  a  certain 
combination  of  strategies. 
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What  is  Game  Theory  [Kollef  ef  ai.,l  9P4); 

*  The  game  theory  allows  mathematical  solutions  of  conflicting 
situations.  It  is  interested  in  situations  in  which  a  Decision  Maker's  (DMs) 
behavior  affects  not  only  their  own  profits  or  losses,  but  also  those  of 
opposing  DMs. 

*  A  strategic  game  consists  of  set  of  players,  strategies  for  each  player  and  payoff  for  each 
strategy  combination.  A  solution  to  a  game  is  a  certain  combination  of  strategies. 


What  is  Game  Theory? 


*  If  there  is  an  interactive  situation  is  described  as  a 
game,  a  formal  analysis  should  find  optimal  strategies  for 
players  and  determine  an  expected  outcome  of  game. 

*  A  solution  to  a  game  is  a  certain  combination  of 
strategies.  This  combination  is  a  Nash  equilibrium  if  it  is  self 
enforcing,  no  player  can  gain  by  unilaterally  deviating 
from  it. 
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What  is  Game  Theory? 

*  If  there  is  an  interactive  situation  is  described  as  a  game,  a  formal 
analysis  should  find  optimal  strategies  for  players  and  determine  an 
expected  outcome  of  game. 

*  A  solution  to  a  game  is  a  certain  combination  of  strategies.  This 
combination  is  a  Nash  equilibrium  if  it  is  self  enforcing,  no  player  can 
gain  by  unilaterally  deviating  from  it. 
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BM  Fuzzy  TOPSIS  (Chen  and  Hwang ,  1992) 

*  Fuzzy  TOPSIS  (The  Technique  for  Order  Preference  by 
Similarity  to  Ideal  Solution)  is  a  multi-criteria  method  to 
identify  solutions  from  a  set  of  alternatives* 

*  The  chosen  alternative  should  have  the  shortest 
distance  from  the  positive  ideal  solution.  According  to  the 
concept  fuzzy  positive  ideal  solution  (FPIS)  and  the  fuzzy 
negative  ideal  solution  (FNIS)  are  calculated.  Vertex 
method  is  applied  to  calculate  the  distance  for  each 
alternative  from  FPlS  and  FNIS.  respectively. 
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Fuzzy  TOPSIS; 

*  Fuzzy  TOPSIS  [The  Technique  for  Order  Preference  by  Similarity  to  Ideal 
Solution)  is  a  multi-criteria  method  to  identify  solutions  from  a  set  of 
alternatives. 

*  The  chosen  alternative  should  have  the  shortest  distance  from  the 
positive  ideal  solution.  According  to  the  concept,  fuzzy  positive  ideal 
solution  (FPIS)  and  the  fuzzy  negative  ideal  solution  (FNIS)  are  calculated. 
Vertex  method  is  applied  to  calculate  the  distance  for  each  alternative 
from  FPIS  and  FNIS,  respectively. 


Fuzzy  TOPSIS 


*  Finally,  closeness  coefficients  of  each  alternative  are 
defined  to  determine  the  ranking  order  of  all 
alternatives. 

*  In  this  case  Fuzzy  TOPSIS  is  used  to  find  implication 
power  values  of  strategies  and  to  form  decision 
payoff  matrix. 
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Fuzzy  TOPSIS  ; 

*  Finally,  closeness  coefficients  of  each  alternative  are  defined  to 
determine  the  ranking  order  of  all  alternatives, 

*  In  this  case  Fuzzy  TOPSIS  is  used  to  find  implication  power  values  of 
strategies  and  to  form  decision  payoff  matrix. 
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B  Steps  of  Fuzzy  TOPSIS 

Step  7.  Coicuiotion  of  aggregated  fuzzy  weight  of  criterion 
and  aggregated  fuzzy  rating  of  alternatives  under  criterion. 

SJeo  2,  Construction  of  the  fuzz /-decision  and  the  normalized 
fuzzy-decision  matrix. 

Step  3,  Construction  of  the  weigh  fed  normalized  fuzzy  decision 
matrix. 

Step  4.  Determining  fuzzy  positive  ideal  solution  I'FPiSj  and  the 
fuzzy  negative  I  deal  solution  (FNIS) 

Step  5.  Calculating  the  distance  of  each  alternative  from  FPIS 
and  FNIS. 

Step  0.  Calculating  the  closeness  coefficients  of  alternatives, 
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Steps  of  Fuzzy  TOPSIS  are; 

Sreo  1.  Co/ci^afror?  of  aggregated  fuzzy  weight  of  criterion  and  aggregated  fuzzy  rating  of 
alternatives  under  criterion. 

Sven  2.  Construction  of  the  fuzzy-decision  and  the  normalized  fuzzy-decision  matrix. 

Step  3.  Construction  of  the  weighted  normalized  fuzzy  decision  matrix. 

Step  4 .  Determining  fuzzy  positive  ideal  solution  (FPtS)  and  the  fuzzy  negative  ideal  solution  { FNtS ) 
Step  5.  Calculating  tfre  distance  of  each  alternative  from  FPiS  and  FNIS. 

Step  6.  Calculating  f/re  closeness  coefficients  of  alternatives. 
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Strengths  and  Weaknesses  of 
Game  Theory 


■  Decision  making  environment  is  so  complex  and 
imprecise, 

*  The  most  difficult  part  is  to  estimate  opposite's 
strategies  and  intentions  to  use  them, 

■  You  hove  to  put  yourself  on  opposite  site  and  make 
evaluations. 

*  And  most  difficult  part  is  to  convert  evaluations  to 
numeric  form  and  form  the  decision  matrix. 
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Strengths  and  Weaknesses  of  Game  Theory; 

*  Decision  making  environment  is  so  complex  and  imprecise. 

*  The  most  difficult  part  is  to  estimate  opposite's  strategies  and 
intentions  to  use  them. 

*  You  have  to  put  yourself  on  opposite  site  and  make  evaluations. 

*  And  most  difficult  part  is  to  convert  evaluations  to  numeric  form  and 
form  the  decision  matrix. 
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1.  Yager,  R.R.,  L'Muftiple  Objective  Decision-Making  Using  Fuzzy  Sets",  Internationa:  Journal  of  Man-Machine  Studies, 
1S77. 

2.  G  odet  M  ,r  Scenarios  ond  Strategic  Management;  London  Euttenvorth,  1BB7. 

5.  Chen  5.  J.  and  Hwang  C.  L_,  Fuzzy  Multiple  Attribute  Decision  Making:  Methods  and  Applications.  Springer,  New 
York, 1992. 

4.  Ross,  T.r  Fuzzy  Logic  wit  h  Engines  ring  Appl  cations  2  nd  Ed .  John  Willey  a  So  ns  Ltd.,  West  Sussex,  Eng  la  nd,  2004. 

5.  feller  D.,  Meg ddo  N .  and  Stengel  E.,  Fast  algorrt hms  for  fi ndi ng  randomized  strategies  i n  game  trees,  Proceedi ngs  of 
26th  Annual  ACM  Symposium  on  Theory  of  Computing,  ACMr  New  Yarik,  1994. 

6.  Apkk,  H.S.  “Fuzzy  Logic  Eased  Game  Theory  Applications  in  Decision  Making  Process1',  Ph.D.  Thesis,  Gazi  University, 
Institute  of  Science  a  nd  Technology  2010. 

7.  Aplak,  H.S.  and  Tiirkbey  O.,  '"Fuzzy  logic  based  game  theory  applications  in  multi-criteria  decision  making  process*, 
Journal  of  Intelligent  and  Fuzzy  Systems,  2013. 
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-  Before  doing  baffle,  in  fhe  lemple  one  calculates  and 
will  win,  because  many  calculations  were  made; 

-  Before  doing  battle,  in  the  temple  one  calculates  and 
will  notwin,  because  few  calculations  were  made; 

-Many  calculations,  victory,  few  calculations,  no  victory, 
then  how  much  less  so  when  no  calculations? 

Sun-Tiu;The  Principles  af  Warfare  "The  Art  of  War" 
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-  Before  doing  battle,  in  the  temple  one  calculates  and  will  wmr  because  many  calculations 
were  made; 

-  Before  doing  battle,  in  the  temple  one  calculates  arid  will  not  win,  because  few 
calculations  were  made; 

-  Many  calculations,  victory,  few  calculations,  no  victory,  then  how  much  less  so  when  no 
calculations? 

Sun-Tzu:  The  Principles  of  Warfare  J,'The  Art  of  War" 
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A.2.5  Improving  Situational  Awareness  through  Patterns 

INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Correlation  Analysis:  Looking  for  Pattern  and  Relationships 

Faculty 

Name(s) 

Carsten  Lauenroth,  Bundeswehr  Planning  Office,  Ottobrunn,  DEU 

Length 

30  -  45  minutes 

Learning 

Objectives 

Understand  the  relationship  between  time  series  analysis  and  correlation 
analysis. 

Appreciate  the  difficulties  of  descriptive  statistics. 

Appreciate  smoothing,  correlation  analysis,  and  causality. 

Appreciate  the  strengths  and  weaknesses  of  quantitative  assessment 
methods. 

Be  able  to  “read”  scatter  plot  charts. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Appreciate  the  diversity  of  data  analysis  and  finding  of  relationships. 

Understand  the  shortcomings  of  different  influences  on  time  series  analysis. 

Content 

Outline 

Case  Study:  Enemy  Fighting  Season  in  AFG. 

Time  Series  Analysis  examples. 

Correlation  Analysis  examples. 

Usage  of  Ratio  and  Smoothing. 

Materials 

List  Required  Pre-Readings: 

ISAF  Monthly  Data:  http://www.isaf.nato.int/article/news/monthly-trends.html. 

Methods 

90%  Lecture  10%  Class  Exercises. 

%  Break  Outs. 

Exercise(s) 

Seminar  participants  will  be  asked  to  evaluate  charts  from  the  case  study. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

In  your  experience,  where  is  data  analysis  used  most  in  your  organization? 

How  can  the  ability  of  time  series  analysis  and  finding  patterns  be  taken  into 
account  while  planning  operations? 

How  can  we  measure  progress/success  in  a  population  centric  COIN 
operation? 
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Improve  Situational 
Awareness 


A  Case  Study  in  Correlation  Analysis 
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Overview 


Situation:  Afghanistan  2011,.  continuous  assessment  on  enemy 
initiated  attacks  at  HQ  15  A  F  A  AG  in  201  L 


Decision:  identify  and  clarify  of  enemy  seasonal  activities  for 
Afghanistan,  generally. 

Objective:  Enhance  situational  awareness  in  HQ  ISAF. 


OA  Contribution:  Time  series  analysis  based  on  historical 
data  and  -show  of  dependencies  of  various  Factofs  {Statistics  [. 

Result;  Model  of  the  enemy  fighting  cycle 
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S  Situation 

*  ISAF,  current  violence  analysis  relies  on  a 
presentation  of  current  numbers  and 
graphs,  which  shows  the  violence  level  at 
once  to  illustrate  useful  findings. 
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Start  with  a  Chart.  This  chart  is  part  of  the  ISAF  A  AG  monthly  data  release  to  the 
press. 

The  graph  represents  the  nationwide  enemy-initiated  attacks  since  January  2008. 
AAG  provides  the  Staff  of  HQ  ISAF  with  violence  analysis  and  data  assessments. 
The  current  violence  analysis  relies  on  a  presentation  of  current  numbers  and 
graphs,  which  shows  the  violence  level  at  once  to  illustrate  useful  findings.  Further 
onF  these  time  series  graphs  include  trends  and  seasonal  variations.  There  are 
many  factors  that  have  to  be  taken  into  account;  the  strength  of  ISAF,  the  change  of 
the  seasons,  the  poppy  cycle,  and  the  enemy  fighting  season. 

Based  on  historical  data,  the  time  series  of  enemy-initiated  attacks  shows  strong 
seasonal  rhythm  with  an  increase  of  enemy-initiated  attacks  during  the  first  half  of 
the  year  and  a  decrease  during  the  second  half.  However,  which  factors  are 
influencing  this  pattern  and  is  it  possible  to  identify-  the  insurgent  fighting  season? 

Definition:  Enemy-initiated  attacks  comprise  enemy  action  (enemy-initiated  direct 
fire,  indirect  fire,  surface-to-air  fire)  and  explosive  hazard  events,  to  include 
executed  attacks  only  (improvised  explosive  device  [I ED)  explosions  /  mine  strikes). 

Data  Source:  Afghan  Mlission  Network  (AMN)  Combined  Information  Data  Network 
Exchange  (CIDNE)  Database,  as  of  16  Feb  2012. 


STO-TR-SAS-098 


ANNEX  -  81 


ANNEX  TO  SAS-098 


organization 


Approach 

*  Question  to  Answer:  How  does  the  enemy 
fighting  cycle  look  and  which  impact  will 
have  various  criteria? 

*  Testing  by  the  OA:  seeking  evidence  by 
collecting  and  analyzing  data  ->  providing 
evidence  for  dependencies  of  various 
relevant  factors 
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R  e  I  eva  nt  fa  ctors  a  re : 

•ISAF  Troop  strength? 

•Temperature? 

•  Rainfall 

•  Snow  level? 

•Poppy  cultivation? 

•Religious  Dates? 

Factors  that  could  have  some  influence  on  the  enemy  fighting  rhythm  are  weather 
conditions,  like  temperature,  rainfall  and  snowfall;  religious  dates,,  like  Ramadan,  Eld  a  I  Fitr, 
and  Eid  a  I  Adlha;  special  events  or  announcements,  like  Mullah  Omar's  Al  Badr 
announcement  or  elections;  the  poppy  cycle;  and  the  ISAF  Troop  level.. 
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3  OA  Contribution 

■  Collecting  the  Data 

o  During  the  time  period  of  Interest,  how  many 
enemy  initiated  attacks  occurred  in  ISAF? 
o  Result:  Based  on  the  data,  these  time  series 
graphs  include  trends  and  seasonol  variations . 
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The  seasonal  adjusted  enemy-initiated  attacks  as  represent  In  upper  chart  shows  also  the 
pattern  of  enemy  activities  over  the  time.  Further  on,  the  graph  shows  the  increase  of 
enemy-initiated  attacks  from  ZOOS  to  2010  and  a  decrease  form  2010  to  2011.  However, 
independent  of  the  level  of  enemy-initiated  attacks  the  rhythm  for  each  year  seems  to  be 
similar. 


A  RIM  A  models  are  the  most  general  class  of  models  for  forecasting  a  time  series.  In  factp 
the  easiest  way  to  think  of  ARIM  A  models  is  as  fine-tune  versions  of  random-walk  and 
random-trend  models.  The  acronym  ARIMA  stands  for  "Auto- Regressive  Integrated  Moving 
Average".  In  this  case  we  used  SARIMA  (1,  lr  2)(1,  1,  2)  52  by  JMP,  a  mixed  model 
(exponential  smoothing  and  autoregressive  Model  which  includes  seasonality). 
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OA  Contribution 
Providing  Evidence 


*  Correlation  Analysis 

o  Ratio  El  A  per  10.000  ISAF  Troops 
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CONCLUSION:  No  change  for  the  pattern! 
However,  the  troop  level  influence  only  the 
level  of  EIA. 
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ARIMA  models  are  the  most  general  class  of  models  for  forecasting  a  time  series.  In  fact, 
the  easiest  way  to  think  of  ARIMA  models  is  as  fine-tune  versions  of  random -walk  and 
random-trend  models.  The  acronym  ARIMAstandsfor  ‘‘Auto- Regressive  Integrated  Moving 
Average".  In  this  case  we  used  SARIMA  (1,  lr  2)  (1#  1,  2)  52  by  JMP,  a  mixed  model 
(exponential  smoothing  and  autoregressive  Model  which  includes  seasonality). 
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OA  Contribution 

Providing  Evidence  for 
Relationships 
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Slide  7  (Correlation  Analysis  (Scatter  Plot)): 

The  relationship  between  the  ISAF  Troop  level  and  the  number  of  enemy-initiated  attacks 
(upper  left  chart)  shows  a  strong,  positive  correlation  between  these  two  factors;  the 
correlation  coefficient  is  0.74  with  p-value  lower  than  0.0001.  That  means  this  result  is 
statistically  significant. 

The  relation  between  the  number  of  enemy-initiated  attacks  and  the  temperature  is 
depicted  in  the  upper  right  chart.  This  chart  shows  the  medium,  positive  correlation 
between  these  two  factors,  for  which  the  correlation  coefficient  is  0.49  with  p-value  lower 
than  0.0001.  That  means;  if  the  temperature  is  increasing  than  the  number  of  enemy- 
initiated  attacks  normally  increases  and  this  is  statistically  significant. 

Additionally,  the  relationship  between  the  number  of  enemy-initiated  attacks  and  the 
rainfall  level  is  shown  in  Figure  3.  This  figure  shows  a  weak,  negative  correlation  between 
these  two  factors;  the  correlation  coefficient  is  -0.34  with  p- value  lower  than 
0.0001[Regarding  the  normal  quantile  plot,  the  num  bers  enemy-initiated  attacks  are 
normal  distributed  within  a  90  percent  confidence  interval].  That  means;  if  the  rainfall  level 
is  decreasing  than  usually  the  number  of  enemy-initiated  attacks  increases  and  this  is 
statistically  significant. 


STO-TR-SAS-098 


ANNEX  -  85 


ANNEX  TO  SAS-098 


organization 


OA  Contribution 
Providing  Evidence  for 
Causalities 
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In  common  usage,  causality  is  also  the  relationship  between  a  set  of  factors  (causes)  and  a 
phenomenon  (the  effect).  Anything  that  affects  an  effect  is  a  factor  of  that  effect.  A  d  irect 
factor  is  a  factor  that  affects  an  effect  directly  that  is.  without  any  intervening  factors. 
(Intervening  factors  are  sometimes  called  "intermediate  factors.”). 
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Conclusions 

Model  of  the  Enemy  Fighting  Cycle  (based  on 
Statistics) 

CAREFUL:  Special  Events  could  have  a  big 
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Conclusions. 

The  insurgent  fighting  cycle  follows  seasonal  pattern! 

This  fighting  cycle  depends  on  va  rious  factors  like  the  temperature,  rainfall,  religious  dates, 
and  the  poppy  cycle.  Based  on  historical  data,  the  insurgent  fighting  cycle  shows  typically 
peaked  during  the  month  of  August  with  violence  levels  tapering  by  September.  The  cycle 
starts  with  the  "Al-Badr"  campaign  in  early  May  and  increase  until  August,  usually  until 
begin  of  the  Ramadan  Period,  and  decrease  up  to  February.  That  will  only  be  interrupted 
for  a  short  increasing  period  after  the  "Eid-Al  Fit rjr  holydays  at  the  end  of  the  Ramadan.  The 
insurgent  attacks  reach  there  lowest  level  in  the  middle  of  February.  During  March  and 
early  April  is  a  short  increase  or  insurgent  attacks  observed,  especially  in  RC  South, 
Southwest,  and  West,  the  main  areas  for  poppy  cultivation  in  Afghanistan.  The  climatology 
has  an  impact  on  insurgent  activities.  We  also  had  shown  that  the  ISAF  Troop  level  has  an 
impact  on  the  number  of  enemy-initiated  attacks.  However,  the  ISAf  Troop  level  does  not 
influence  the  insurgent  fighting  cycle,  generally 
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S  Used  OA  Tools 


Microsoft  Excel: 

*  Commercial  Spreadsheet  Application 
■  Strengths: 

*  Standard  Software 

*  Calculation,  Graphing  Tools,  PivotTables,  and 
Macro  Programming  (VBA) 

*  Weakness: 

*  Accuracy  and  Convenience  of  Statistical  Tools 
(handling  of  round-off  and  large  numbers] 
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Jtnp 
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JMP: 

*  Commercial  Application  to  Perform  Simple  and 
Complex  Statistical  Analysis 

•  Sfrengfhs: 

■  Statistical  Software 
■  Weakness; 

■  No  Standard  Software 
*  Needs  Training 
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HH  Summary 

*  Without  evidence  provided  by  OA  there 
might  have  been  no  thorough  analysis  ot 
the  enemy  fighting  pattern 

*  To  best  utilize  your  OAs  to  oid  in  decision 
making,  they  need  access  to  you,  the 
decision  maker. 

*  The  OA  brings  a  variety  of  tools  like  optimiza¬ 
tion,  simulation,  statistics  (e.g.  hypothesis 
testing),  and  assessment  skills  to  help  you 
with  evidence  based  decision  making. 
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S  ISAF  Enemy-Initiated  Attacks 
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Definition:  Enemy-initialed  attacks  comprise  enemy  action  (enemy-initiated  direct  fine,  indirect  fire, 
surface-to-air  fire)  and  explosive  hazard  events,  to  include  executed  attacks  only  (improvised 
explosive  device  (IED)  explosions  y  mine  strikes). 

Data  Source:  Afghan  Mission  Network  (AMN)  Combined  Information  Data  Network  Exchange 
(Cl ONE)  Database,  as  of  1 6  Feb  2012. 
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ARIMA  models  are  the  most  general  class  of  models  for  forecasting  a  time  series.  In  fact, 
the  easiest  way  to  think  of  ARIMA  models  is  as  fine-tune  versions  of  random-walk  and 
random-trend  models.  The  acrony  m  ARIMA  stands  for  "Auto- Regressive  Integrated  Moving 
Average".  In  this  case  we  used  SARIMA  (1,  lr  2)  (1,  1,  2)  52  by  JMP,  a  mixed  model 
(exponential  smoothing  and  autoregressive  Model  which  includes  seasonality). 

Definition:  Enemy-initiated  attacks  comprise  enemy  action  {enemy-initiated  direct  fine,  indirect  fire, 
surface-to-air  Hre} and  explosive  hazard  events,  to  include  executed  attacks  only  (improvised 
explosive  device  (IED)  explosions  /  mine  shrikes). 

Data  Source:  Afghan  Mission  Network  fAMN)  Combined  Information  Data  Network  Exchange 
(CIDNE)  Database,  as  of  1 6  Feb  2012. 
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B  Impact  of  ISAF  Troop  Level 
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As  shown  on  the  slide  before;  over  the  last  4  years  the  seasonal  pattern  has  remained 
unchanged,  as  only  the  level  of  enemy-initiated  attacks  increased.  For  that  reason,  we 
computed  the  ratio  of  enemy-initiated  attacks  per  10,000  ISAF  troops  {depleted  on  this 
slide).  This  graph  shows  the  same  pattern  of  enemy  activity  over  the  last  4  years.  Given 
this,  we  can  assess  there  is  no  impact  on  the  fighting  season  pattern  of  the  insurgent.  The 
ISAF  troop  level  has  only  an  influence  on  the  level  of  violence. 

ARIMA  models  are  the  most  general  class  of  models  for  forecasting  a  time  series.  In  fact, 
the  easiest  way  to  think  of  ARIMA  models  is  as  fine-tune  versions  of  random -walk  and 
random-trend  models.  The  acronym  ARIMA  stands  for  "Auto- Regressive  Integrated  Moving 
Average''.  In  this  case  we  used  SARIMA  (1,  lr  2}  (1,  1,  2)  52  by  JMP,  a  mixed  model 
(exponential  smoothing  and  autoregressive  Model  which  includes  seasonality). 

Definition:  Enemy4nitiated  attacks  comprise  enemy  action  (enemy-initiated  direct  fine,  indirect  fire, 
surface-to-air  ire)  and  explosive  hazard  events,  to  include  executed  attacks  only  (improvised 
explosive  device  ( IE D) explosions.!1  mine  strikes). 

Data  Source:  Afghan  Mission  Network  i'AMN)  Combined  Information  Data  Network  Exchange 
{CIDNEj  Database,  as  of  1 6  Feb  2012.  ' 
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Relationships: 

Correlation  Analysis  (Scatter 
Plot) 


*  Statistical  relationship  between  two  sets  of  data 

*  Correlation  coefficient  -  measuring  the  degree  of 
correlation 
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Short  [introduction  of  Correlation  Analysis  and  Scatter  Plots. 

Correlation  Analysis: 

In  statistics,  dependence  refers  to  any  statistical  relationship  between  two  sets  of  data. 
Correlation  refers  to  any  of  a  broad  class  of  statistical  relationships  involving  dependence. 
There  are  several  correlation  coefficients,  often  denoted  p  or  r,  measuring  the  degree  of 
correlation.  The  most  common  of  these  is  the  Pearson  correlation  coefficient,  which  is 
sensitive  only  to  a  linear  relationship  between  two  variables. 

In  the  case  of  the  bivariate  normal  distribution  the  population  Pearson  correlation 
coefficient  characterizes  the  joint  distribution  as  long  as  the  marginal  means  and  variances 
are  known.  For  most  other  bivariate  distributions  this  is  not  true.  Nevertheless,  the 
correlation  coefficient  is  highly  informative  about  the  degree  of  linear  dependence 
between  two  random  quantities  regardless  of  whether  their  joint  distribution  is  normal. 
The  result  is  statistically  significant. 

[In  statistical  significance  testing,  the  p-value  is  the  probability  of  obtaining  a  test  statistic 
at  least  as  extreme  as  the  one  that  was  actually  observed,  assuming  that  the  nul  l 
hypothesis  is  true.  One  often  '"rejects  the  null  hypothesis11  when  the  p-value  Is  less  than  the 
significance  level  a  (Greek  alpha),  which  is  often  0.05  or  0.01.  When  the  null  hypothesis  is 
rejected,  the  result  is  said  to  be  statistically  significant! 

For  uncentered  data,  the  correlation  coefficient  corresponds  with  the  cosine  of  the  angle 
between  both  possible  regression  lines  y=gx[x)  and  a=gy(y). 
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Relationships: 

Correlation  Analysis  (Scatter 
Plot) 


El  A  versos  ISAF  Troops  strength 
^  Strong  positive  Relation  shi  p  (r  = 
^  p-value:  <  Q.QOQi 
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Du  ring  the  last  couple  of  years,  ISAF  troop  levels  rose  to  approximately  120r000  troops. 
Due  to  the  increasing  ISAF  operations  tempo  the  number  of  enemy-initiated  attacks 
increased  significant  over  that  same  time.  However,  has  the  ISAF  troop  level  had  an  effect 
on  the  enemy  fighting  cycle? 

The  relationship  between  the  ISAF  Troop  level  andt  he  number  of  enemy-initiated  attacks 
shows  a  strong,  positive  correlation  between  these  two  factors;  the  correlation  coefficient 
is  0.74  with  p-value  lower  than  0.0001.  That  means  this  result  is  statistically  significant. 
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B  Relationships: 
Temperature 
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Afghanistan  has  an  arid,  semi-arid  to  sub-humid  climate  with  high  inter-annual  variation  of 

precipitation.  The  chart  shows  the  recorded  tem  perature  for  five  different  regions  of 
Afghanistan  since  September  2007.  The  temperature  typically  starts  decreasing  in 
September  and  drops  to  freezing  m  December  in  the  Central  Highlands  and  north-eastern 
parts  and  continues  to  decline  until  February.  Generally,  January  is  the  coldest  month  of 
the  year.  The  temperatures  increase  in  March  and  reach  maxi  mums  in  July  and  August. 
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Relationships: 

Correlation  Analysis  (Scatter 


Plot) 


EPA  versus  Temperature' 

^  Medium  positive  (Relationship  (r  -  G.4t) 
^  p-^alue:  £  O.OOftl 
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The  reflation  between  the  number  of  enemy-initiated  attacks  and  the  temperature  is 
depicted  in  the  chart.  This  chart  shows  the  medium,  positive  correlation  between  these 
two  factors,  foir  which  the  correlation  coefficient  is  0.49  with  p-value  lower  than  0.0001. 
That  means;  if  the  temperature  is  increasing  than  the  number  of  enemy-initiated  attacks 
normally  increases  and  this  is  statistically  significant. 
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Relationships: 
Rainfall  Level 
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Spatial  distribution  of  rainfall  is  variable  in  different  regions  of  the  country.  During  the  rainy 
season  the  most  amount  of  rainfall  occurred  in  the  Eastern  region,  and  the  north -eastern, 
northern,  north-western,  capital  and  eastern  parts  of  the  Central  Highlands  experienced 
moderate  rainfall  The  southern,  south-western  and  western  regions  usually  receive  low 
amounts  of  rainfall. 

At  the  beginning  of  the  rainy  season  snow  starts  in  some  parts  of  the  country.  During 
October,  snow  pack  develops  in  most  parts  of  the  country,  particularly  in  the  Central 
High  lands  and  north-eastern  region,  during  December  and  mid-January  snow  is  light  an  d 
early  January  snow  increases  in  most  parts. 
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Relationships; 

Correlation  Analysis  (Scatter 
Plot) 
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Additionally,  the  relationship  between  the  number  of  enemy-initiated  attacks  and  the 
rainfall  level  is  sho  wn  in  Figure  S.  This  figure  shows  a  weakr  negative  correlation  between 
these  two  factors;  the  correlation  coefficient  is  -0.34  with  p-value  lower  than 
0.0001[Regarding  the  normal  quantile  plot,  the  numbers  enemy-initiated  attacks  are 
normal  distributed  within  a  90  percent  confidence  interval].  That  means;  if  the  rainfall  level 
is  decreasing  than  usually  the  number  of  enemy-initiated  attacks  increases  and  this  is 
statistically  significant. 
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Ramadan  is  the  holy  ninth  month  of  the  Islamic  calendar  occurring  10-11  days  earlier  each 
successive  year  on  the  Gregorian  calendar.  In  2008,  Ramadan  began  on  01  September  and 
ended  on  30  September;  in  2009 r  Ramadan  began  on  21  August  and  ended  on  19 
September;  in  2010,  Ramadan  fell  between  11  August  and  09  September.  In  2011,  this 
month-long  religious  observance  fell  between  01  and  29  August,  according  to  official  dates 
set  by  religious  leaders  in  Saudi  Arabia. 


In  2008,  there  was  an  average  of  36  enemy-initiated  attacks  per  day  during  Ramadan, 
compared  to  an  average  of  42  attacks  per  day  in  the  30  days  prior  to  Ramadan,  a  14% 
decrease  consistent  with  the  seasonal  decline.  In  2009,  attack  levels  during  Ramadan 
decreased  16%  from  the  30  days  prior.  Similarly,  enemy-initiated  attacks  in  2010  did  not 
show  any  significant  change  in  levels  of  insurgent  attacks  during  Ramadan.  However, 
Ramadan  fell  in  the  pre-election  period  of  the  Parliamentary  Elections  on  IS  September 
2010.  Also,  2011  did  not  show  any  significant  change  in  the  level  of  attacks  {shown  on  this 
slide). 

The  chart  also  shows  bands  corresponding  to  the  "al-Badr"  announcements  and  the  first 
period  of  the  Taliba  n  campaign  from  2008  through  2011.  In  2009  and  2010,  enemy- 
initiated  attacks  increased  by  approximately  45  percent  in  the  four  weeks  following  the 
announcement  of  the  Taliban  offensive.  In  2011,  enemy-initiated  attacks  increased  only  by 
IS  percent  due  to  the  warm  winter  conditions  where  the  level  of  insurgent  attack  was 
higherthan  the  years  before. 
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Conclusions: 

The  insurgent  fighting  cycle  follows  seasonal  pattern.  This  fighting  cycle  depends  on  various 
factors  like  the  temperature,  rainfall,  religious  dates,  and  the  poppy  cycle.  Based  on 
historical  data,  the  insurgent  fighting  cycle  has  typically  peaked  during  the  month  of  August 
with  violence  levels  tapering  by  September.  The  cycle  starts  with  the  "al-Badr"  campaign  in 
early  May  and  increase  until  August,  usually  untiiil  begin  of  the  Ramadan  period,  and 
decrease  up  to  February.  That  will  only  be  interrupted  for  a  short  increasing  period  after 
the  Eid-al  Fitr  holy  days  at  this  end  of  Ramadan.  The  insu  rgent  attacks  reach  there  lowest 
level  in  the  middle  of  February.  During  March  and  early  April  is  a  short  increase  of  insurgent 
attacks  observed,  especially  in  RC  South,  Southwest  and  West,  the  main  areas  for  poppy 
cultivation  in  Afghanistan.  This  paper  shows  that  several  factors  have  more  or  less  influence 
on  the  insurgent  fighting  behaviour.  So  has  the  climatology  an  impact  on  the  insurgent 
activities.  We  also  had  shown  that  the  ISAF  troop  level  has  an  impact  on  the  number  of 
enemy-initiated  attacks.  However,  the  troop  level  does  not  influence  the  insurgent  fighting 
pattern,  generally. 
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Recommended  readings: 

•Why  operations  assessments  fall  (Dr.  Jonathan  Schroden,  CNA) 

•Operations  Assessment  in  Afghanistan  is  broken  (Dr.  Stephen  Downes-Martin,  NWC) 
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A.2.6  Improving  Situational  Awareness  through  Descriptive  Statistics 

INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Descriptive  Statistics:  Hotspot  Analysis  and  Geospatial  Findings 

Faculty 

Name(s) 

Carsten  Lauenroth,  Bundeswehr  Planning  Office,  Ottobrunn,  DEU 

Length 

30  -  45  minutes 

Learning 

Objectives 

Understand  the  relationship  between  time  series  analysis  and  geospatial 
analysis. 

Appreciate  the  difficulties  of  using  descriptive  statistics. 

Appreciate  the  strengths  and  weaknesses  in  using  ratios. 

Appreciate  survey  data  and  geospatial  methods. 

Be  able  to  “read”  box  plots. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Appreciate  the  diversity  of  ratio  analysis  and  geospatial  methods. 

Understand  the  shortcomings  of  different  constraints  on  data  visualization 
and  time  series  analysis. 

Content 

Outline 

Case  Study:  Violence  Trend  on  Province  Level  in  AFG. 

Box  Plot  examples. 

Geospatial  Analysis  approach. 

Usage  of  Survey  Data. 

Materials 

List  Required  Pre-Readings: 

ISAF  Monthly  Data:  http://www.isaf.nato.int/article/news/monthly-trends.html. 

Methods 

90%  Lecture  10%  Class  Exercises. 

%  Break  Outs. 

Exercise(s) 

Seminar  participants  will  be  asked  to  evaluate  box  plots  from  the  case  study. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

In  your  experience,  where  is  descriptive  statistics  used  most  in  your 
organization? 

How  can  the  outcomes  of  a  trend  analysis  taken  into  account  during  planning 
operations? 

How  can  we  measure  progress/success  in  a  population  centric  COIN 
operation? 
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A  Case  Study  in  Descriptive  Statistics 
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Overview 

Situation:  Afghanistan  201 1,  continuous  assessment  on  enemy 
initiated  attacks  at  HQ  ISAF  A  AG  in  2011.  Permanent  task  by 
COM  ISAF. 

Decision:  Identify  and  clarify  progress  in  RC  EAST  AQO, 

Objective:  Enhanced  situational  avvarene&s  in  HQ  ISAF, 

OA  Contribution:  Time  series  analysis-  based  on  historical  data, 
geospatial  analysis  and  survey  findings. 

Outcome:  situation  report  -  information  paper. 
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Overview 

Situation:  Afghanistan  2011,  continuous  assessment  on  enemy 
initiated  attacks  at  !HQ  ISAF  A  AG  In  201 1 .  Permanent  task  by 
COM  ISAF, 

Decision:  Identify  and  clarify  progress  in  PC  EAST  AQD, 

Objective:  Enhanced  situational  avvarene&s  in  HQ  ISAF. 

OA  Contrl  button:  Time  series  analy^fe  based  on  historical  data, 
geospatial  analysis,  and  survey  findings. 

Outcome:  sikiafian  report  -  information  paper. 
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Start  with  a  Chart.  This  chart  is  part  of  the  ISAF  AAG  monthly  data  release  to  the  press. 

The  graph  represents  the  year  over  year  change  of  enemy-initiated  attacks  in  RC  East  since  January 
2D07.  In  addition,  the  charts  shows  that  39  percent  of  the  nationwide  EIA  are  happened  in  RC  East 
during  a  period  of  12  month. 

AAG  provides  the  Staff  of  HG  ISAF  with  violence  analysis  and  data  assessments.  Recent 
Combined  Team  and  Coalition  Force  successes  in  the  southern  portions  of  Afghanistan 
have  led  to  a  steady  and  progressive  decrease  in  levels  of  significant  activity  (SIGACTs)  and 
violence.  However,  the  situation  in  RC  east  is  different.  The  differences  are  not  only  the 
increasing  of  security  incidents,  also  in  particular  the  terrain  and  the  kinds  of  attacks  are 
different.  Friendly  forces  movements  are  only  possible  along  the  small  vafleys,  the 
mountainous  area  offer  the  insurgents  various  hiding-places.  Further  on,  the  far  eastern 
part  of  the  region  is  one  of  the  poorest  areas  of  Afghanista  n  due  to  a  lack  of  infrastructure. 
Current  cross-border  fires  into  Afghanistan  as  a  response  to  imminent  threats  from 
insurgents  safe  havens  in  Kunar  and  Nuristan.  The  Pakistan  Army  reinforced  their  troops 
and  operates  with  25r000  troops  along  the  border  in  the  area  of  Nangarhar,  Nuristan  and 
Kunar  (N2K). 


Definition:  Enemy-initiated  attacks  comprise  enemy  action  (enemy-initiated  direct  fire,  indirect  fire, 
surface-to-air  fire}  and  explosive  hazard  events,  to  include  executed  attacks  only  (improvised 
explosive  device  (IIED)  explosions  i  mine  strikes). 

Data  Source:  Afghan  Mission  Network  (AMN)  Combined  Information  Data  Network  Exchange 
(CIDNE)  Database,  as  of  1 6  Feb  2012. 
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Approach 


*  Question  to  Answer:  What  makes  the 
situation  different  in  RC  East? 


♦  Testing  by  the  OA:  seeking  evidence  by 
collecting  and  analyzing  data  ->  providing 
evidence  for  dependencies  of  various 
relevant  factors 
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What  makes  the  situation  different  in  RC  East?: 

•  Distribution  of  Enemy-initiated  Attacks? 

•  Terrain? 

•  Distribution  of  Own  Troops? 


I  will  try  to  give  some  answers  to  this  question,  rc  East,  the  most  populated 
region  in  Afghanistan,  has  seen  a  34  percent  decrease  in  El  A  in  Jan  2012  compared  to  Jan 
2011.  This  is  attributed  to  extreme  winter  weather  conditions.  However,,  RC  East  was  the 
most  violent  RC  between  January  2011  and  January  2012. 
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mm  OA  Contribution 

*  Collecting  the  Data 

o  During  the  time  period  of  Interest,  how  many 
enemy  initiated  attacks  occurred  in  RC  east? 
o  Result:  Identifying  the  relation  between  terrain, 
population  and  data  of  enemy-initiated  attacks. 


Geospatial  Analysis 

o  The  distribution  of  enemy- initiated  attacks  shows 
also  the  changes  of  enemy  activities  aver  the 
time  on  the  data. 
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RC  East  is  a  landlocked  mountainous  area  with  the  highest  mountains  in  Nuristan,, 
Nangarharr  Kunar  and  Laghman  (N2KL)  Provinces.  The  terrain  is  a  mixture  of  high  peaks  and 
river  valleys  in  the  east  and  shallow  central  plains.  There  are  only  few  main  routes  along 
the  Kabul  River,  the  main  riiver  in  eastern  Afghanistan  flowing  through  the  city  of  Jalalabad 
into  Pakistan,  and  the  Kunar  River,  which  flows  south  along  the  north -south  axis  of  the 
Kunar  Valley.  The  river  valleys,  surrounded  by  steep  mountains,  are  natural  obstacles 
affecting  all  movements  through  the  region.2  RC  East  has  a  continental  climate  with  very 
harsh  winters,  especially  in  the  glaciated  area  around  Nuristan  Province,  and  hot  summers 
in  the  low-lying  areas  of  the  Kabul  River  basin  in  Jalalabad  in  the  southeast. 

There  are  four  main  supply  routes  (MSRs)  in  the  RC  East  area  of  operations  (AGO), 
including  southern  routes  into  Kabul  Province,  Highway  1  through  Ghazni  and  Wardak 
Provinces,  Route  UTAH  through  Logar,  Paktfka,  Paktiya,  and  Khost  Provinces,  and  Highway  7 
through  from  Kunar  Province  through  Laghman  Province  and  conversely  Nangarhar 
Province  to  Pakistan.  RC  East  shares  a  725-kilometer  border  with  Pakistan's  Northwest 
Frontier  Province  (NWFP)  and  Federally  Administered  Tribal  Areas  (FATA). 

RC  East  has  a  large  quantity  of  mineral  deposits  for  Afghanistan,  e.g.  one  of  the  world's 
richest  deposits  of  lithium  can  be  found  in  Ghazni  Province. 


The  population  in  RC  East  is  approximately  7.5  million  people,  51  percent  of  the  total 
Afghan  population.  The  largest  population  areas  are  in  Nangarhar  (1.4  million)  and  Ghazni 


Provinces  (1.15  million).  The  most  densely  populated  provinces  are  Kapisa  (213  inhabitants 
per  square  kilometer),  Nangarhar  (197  inhabitants  per  square  kilometer)  and  Khost  Provinces 
(131  inhabitants  per  square  kilometer).  Comparatively,  the  population  density  in  Kunar 
Province  is  only  86  inhabitants  per  square  kilometer. 


The  region  Is  compromised  of  various  ethnic  groups;  the  northwest  is  predominantly  Hazara; 
the  area  north  of  Kabul  is  predomina  ntly  Tajik;  and  the  mou  ntainous  regions  along  the 
border  of  Afghanistan  and  Pakistan  is  predominantly  Pashtun  with  the  area  in  and  around 
Nuristan  Province  being  Nuristani. 


Data  Source:  C5Q:  Settled  Population  by  Province  (updated  by  CSG),  as  of  11  August  2011 
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Providing  Evidence 

Descriptive  Statistics 

o  Attempt  to  compare  different  areas: 

*  Density  oF  EIA  per  kmr 
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Using  a  metric,  the  ratio  of  enemy-initiated  attacks  and  the  surface  area  of  the  province., 
can  illustrate  an  additional  perspective  on  how  kinetic  activity  affects  the  population.  The 
density  of  that  ratio  shows  for  Helm  and  Province,  which  is  the  province  with  the  highest 
total  nu  mber  of  enemy- initiated  attacks  (median  of  127  weekly  enemy-initiated  attacks 
since  January  2008)  in  Afghanistan  the  enemy-initiated  attack  level  per  square  kilometer  is 
only  0.22  percent;  Kandahar  Province  with  median  of  enemy-initiated  attacks  of  45  since 
January  2008  and  enemy  initiated-attack  per  square  kilometer  or  0.0S  percent.  Kunar 
Province  in  comparison,  which  has  a  median  of  35  enemy-initiated  attacks  per  week  since 
January  2008,  has  an  enemy-initiated  attack  per  square  kilometer  level  of  0.72  percent. 

This  is  the  highest  nationwide  ratio  of  enemy-initiated  attacks  to  surface  area  of  a  province. 

Data  Sources  Ethnic  distribution  regarding  results  of  the  ANQAR  Survey  (wave  13),  as  of  23 
September  2011. 

The  Lands  can  Global  Population  Project  is  a  worldwide 

population  database  for  estimating  ambient  populations 
at  risk.  Best  available  census  counts  are  distributed  to 
cells  based  on  probability  coefficients  which,  in  turn,  are 
based  on  road  proximity,  slope,  land  cover,  and 
nighttime  lights. 

Kinetic  activity  is  defined  as  primary  type:  {enemy  action, 
explosive  hazard,  friendly  action};  primary  type:  {direct 
fire,  indirect  fire.,  IED  /  mine  strike,  surface-to-air  fire} 
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c  Attempt  to  compare  different  areas: 

•  Population  dentily  and  El  A 
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Using  Landscan  2009  data  to  estimate  population  and  population  density,  we  can  better 
understand  the  number  and  distribution  of  Afghans  affected  by  violence.  The  national  per 
capita  violence  level  is  5.9  events  per  10,000  inhabitants  for  the  period  March- August 
2011.  By  province  there  are  wide  variations.  Helmand  Province,  which  accounts  for  3.7 
percent  of  the  population  and  32  percent  of  all  violence;  Kandahar  Province,  which 
accounts  for  4.5  percent  of  the  population  and  15  percent  of  all  kinetic  activity,  and  Kunar 
Province,  which  accounts  for  1.6  percent  of  the  population  has  S  percent  of  all  violence. 

Data  Sources  Ethnic  distribution  regarding  results  of  the  ANQAR  Survey  (wave  13),  as  of  23 
September  2011. 

The  Lands  can  Global  Population  Project  is  a  worldwide 

population  database  for  estimating  ambient  populations 
at  risk.  Best  available  census  counts  are  distributed  to 
cells  based  on  probability  coefficients  which,  in  turn,  are 
based  on  road  proximity,  slope,  land  cover,  and 
nighttime  lights. 

Kinetic  activity  is  defined  as  primary  type:  {enemy  action, 
explosive  hazard,  friendly  action};  primary  type:  {direct 
fire,  indirect  fire,  IED  /  mine  strike,  surface-to-air  fire} 
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The  rise  of  the  restrictions  of  freedom  of  movement  for  inhabitants  of  the  Kunar  Province  is 
also  reflected  in  an  increase  of  I  ED  explosions  along  the  MSP.  This  slide  shows  the  increase 
and  the  distribution  of  IED  explosions  in  Kunar  for  the  period  from  January  to  August  for 
2003,  2009,  2010,  2011. 


Follow-on  actions 
supported  by  OA 


*  Several  indicators  were  discovered  which 
could  have  indicated  the  perception  of  the 
population: 
o  Survey  trends 
o  Freedom  of  movement? 


StfdeS: 

The  results  from  ANQAR  Survey,  conducted  in  September  2011,  support  this  trend  by 
indicating  a  lack  of  freedom  of  move  ment  in  Kunar  Province. 


ANQAR:  Afghan  Nationwide  Quarterly  Assessment  Research 
Data  Source:  ANQAR  Survey,  as  of  September  .20 11. 
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Conclusions 


*  Important  situation  report  for  RC  EAST 

*  Statistically  support  to  impressions  of  the  PRT 
Kunar 

*  Combination  of  different  methodologies 
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Conclusions. 

Several  factors  contribute  to  the  unique  and  segmented  violence  trends  in  RC  East  - 
disparate  and  highly  distributed  enemy  activity;  highly  restrictive  terrain;  and  the  proximity 
to  Pakistan.  Insurgent  activity  in  RCs  South  and  Southwest  tends  to  be  concentrated  in  key 
populated  areas,  which  provides  better  opportunities  for  control. 

Recent  Combined  Team  and  Coalition  Force  successes  in  the  southern  portions  of 
Afghanistan  have  led  to  a  steady  and  progressive  decrease  in  levels  of  significant  activity 
(SIGACTs)  and  violence.  However,  the  situation  in  RC  east  is  different.  The  differences  are 
not  only  the  increasing  of  security  incidents,  also  in  particular  the  terrain  and  the  kinds  of 
attacks  are  different.  Friendly  forces  movements  are  only  possible  along  the  small  valleys, 
the  mountainous  area  offer  the  insurgents  various  hiding-places.  Further  on r  the  far  eastern 
part  of  the  region  is  one  of  the  poorest  areas  of  Afghanistan  due  to  a  lack  of  infrastructure. 
Current  cross-border  fires  into  Afghanistan  as  a  response  to  im  minent  threats  from 
insurgents  safe  havens  in  Kunar  and  Nuristan.  The  Pakistan  Army  reinforced  their  troops 
and  operates  with  25,000  troops  along  the  border  in  the  area  of  Nangarhar,  Nuristan  and 
Kunar  (N2K). 
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Used  OA  Tools 

•  Microsoft  Excel: 

-  Comrnefdd  Spreadsheet  Applicalion 

*  Strengths: 

■  Standard  Software 

■  Calculation,  Graphing  Tools.  Pivot  Tables,  and  Macro 
Programming  (VEJAJ 

*  W&cikness: 


■  A  cc  u  racy  a  nd  Con  von  ience  of  Sta  tistico  I  T  oofs  ( h  a  ndling 
or  round-off  and  large  numbers) 

ArcGIS: 

*  Commercial  Application  to  work  with  maps  and  geographic 


#Arc 

GIS 


information 

*  Strengths; 

'  Compiling  geographic  data  into  a  single  workspace 
*  Analyzing  mapped  information  using  straight  statistic  and 
geo-staflsflcal  methods 

*  Weakness: 

"  M 1  i  -  N  o  Sto  n  d  a  rd  Software  o  nd  needs  troini  ng 


Used  OA  Tools 


•  ANQAR  Survey: 

-  Most  comprehensive  national  survey; 

*  Quarterly  results  since  SEP  2008 

*  Jun  2011: 12,581  sample  size,  surveyed  in  246  districts  find. 

14  in  Kabul) 

*  Sep  2011:  12P603  sample  sire,  surveyed  in  232  districts  (incl. 

15  in  Kabul] 

*  Strengths:: 

■  Own  Questionnaire 
-  Weakness: 

*  Commercial  Provider 
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ANQ.AR:  Afghan  Nationwide  Quarterly  Assessment  Research 
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S3  Summary 

*  Without  evidence  provided  by  OA  there 
might  have  been  no  thorough  analysis  of 
the  situation  in  RC  East 

*  To  best  utilize  your  OAs  to  aid  in  decision 
making,  they  need  access  to  you,  the 
decision  maker. 

*  The  OA  brings  a  variety  of  tools  like  optimiza¬ 
tion,  simulation,  statistics  (e.g,  hypothesis 
testing),  and  assessment  skills  to  help  you 
with  evidence  based  decision  making, 

IUWJ2  13 


EJH  Population: 

6323  Situation  in  RC  EAST 


RC  W 
RCSW  12ft 
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The  population  in  RC  East  Is  approximately  7.5  million  people,  SI  percent  of  the  total 
Afghan  population.  The  largest  population  areas  are  in  Nangarhar  (1.4  million)  and  Ghazni 
Provinces  (1.15  million),.  The  most  densely  populated  provinces  are  Kaplsa  (2 IS  inhabitants 
per  square  kilometer),  Nangarhar  (197  inhabitants  per  square  kilometer)  and  Khost 
Provinces  (131  inhabitants  per  square  kilometer).  Comparatively,  the  population  density  in 
Kunar  Province  is  only  S 6  inhabitants  per  square  kilometer. 


The  region  is  compromised  of  various  ethnic  groups;  the  northwest  is  predominantly 
Hazara;  the  area  north  of  Kabul  is  predominantly  Tajik;  and  the  mountainous  regions  along 
the  border  of  Afghanistan  and  Pakistan  is  predominantly  Pashtun  with  the  area  in  and 
around  Nuristan  Province  being  Nurlstanl. 


Data  Source:  CSO:  Settled  Population  iby  Province  (updated  by  CSQ)r  as  of  11  August  2011 
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m  Descriptive  Statistics: 
™  Box  Plot 


*  Graphically  depicting  groups  of  numerical 
data 

*  Representation  of  the  distribution  of  data  sets 


□ - 


I  i 
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Short  introduction  of  box  plots. 

In  descriptive  statistics,  a  box  plot  (also  known  as  a  box-and -whisker  diagram)  is  a 
convenient  way  of  graphically  depicting  groups  of  numerical  data  through  their  five- number 
summaries:  the  smallest  observation  (sample  minimum},  lower  ciuartile  (Q1),  median  (Q2), 
upper  quartile  (Q6V.  and  largest  observation  (sample  maximum).  A  box  plot  may  also 
indicate  which  observations,  if  any,  might  be  considered  outliers. 

The  I m  pact  Factor  box  plot  depicts  the  distribution  of  I m  pact  Factors  for  all  journals  in  the 
category.  The  horizontal  lime  that  forms  the  top  of  the  box  is  the  76th  percentile  (Q3).  The 
horizontal  line  that  forms  the  botto  m  is  the  26th  percentile  (Ql).  The  horizontal  line  that 
intersects  the  box  is  the  median  Impact  Factor  for  the  category.  The  cross  represents  the 
mean  value. 

Horizontal  lines  above  and  below  the  box,  called  wfirstiers,  represent  maximum  and 
minimum  values.  The  top  whisker  is  the  smaller  of  the  following  two  values: 

•  the  maximum  Impact  Factor  (IF) 

Q3  IF  +  3.5 (Q3  IF-Q1  IF) 

The  bottom  whisker  is  the  larger  of  the  following  two  values: 

•  the  minimum  Impact  Factor  (IF) 

a3  IF -3.5(03  IF -01  IF) 

An  circle  represents  an  outlier,  which  is  a  value  greater  than  the  value  represented  by  the 
top  whisker.  An  asterisk  represents  a  "far11  outlier,  which  is  a  value  greater  than  the  sum  of 
the  value  represented  by  the  top  whisker  and  5(Q3  IF  -  01  IF). 
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Descriptive  Statistics; 

Ratios,  be  carefully 

Attempt  to  compare  different  areas: 

*  Density  of  El  A  per  km2 


a* 

4 

1- 

!m 

w 

a  iff 


//s/s/  /  ✓  //////// 
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Using  a  metric,  the  ratio  of  enemy-initiated  attacks  and  the  surface  area  of  the  province., 
can  illustrate  an  additional  perspective  on  how  kinetic  activity  affects  the  population.  The 
density  of  that  ratio  shows  for  Helm  and  Province,  which  is  the  province  with  the  highest 
total  number  of  enemy- initiated  attacks  (median  of  127  weekly  enemy-initiated  attacks 
since  January  2008)  in  Afghanistan  the  enemy-initiated  attack  level  per  square  kilometer  is 
only  0.22  percent;  Kandahar  Province  with  median  of  enemy- initiated  attacks  of  45  since 
January  ZOOS  and  enemy  initiated-attack  per  square  kilometer  or  0.0S  percent.  Kunar 
Province  in  comparison,  which  has  a  median  of  35  enemy-initiated  attacks  per  week  since 
January  ZOOS,  has  an  enemy-initiated  attack  per  square  kilometer  level  of  0.72  percent. 

This  is  the  highest  nationwide  ratio  of  enemy-initiated  attacks  to  surface  area  of  a  province. 

Data  Source,  Ethnic  distribution  regarding  results  of  the  AWQAR  Survey  (wave  13),  as  of  23 
September  2011. 

The  Lands  can  Global  Population  Project  is  a  worldwide 

population  database  for  estimating  ambient  populations 
at  risk.  Best  available  census  counts  are  distributed  to 
cells  based  on  probability  coefficients  which,  in  turn,  are 
based  on  road  proximity,  slope,  land  cover,  and 
nighttime  lights. 

Kinetic  activity  is  defined  as  primary  type:  {enemy  action, 
explosive  hazard,  friendly  action};  primary  type:  {direct 
fire,  indirect  fire,  IED  /  mine  strike,  surface-to-air  fire} 
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Descriptive  Statistics: 
Geospatial  Analysis 
Enemy-initiated  attacks  aver  a  period  of  9 
month 


llJW.13 
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This  map  shows  that  enemy-initiated  attacks  are  concentrated  in  Kunar,  Khost,  and  Kapisa 
Provinces  as  well  as  near  the  Ring  Roadl  in  Logar,  Wardak,  and  Ghazni  Provinces, 

Data  Source:  AIM  N  CIDNE 


EEa  Descriptive  Statistics: 
a  HotSpots? 

♦  Enemy-initiated  attacks  over  a  period  of  9 
month 

Laghnun  Par#  an 
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This  map  indicates  the  distribution  of  security  incidents  by  province.  Security  incidents  are 
concentrated  in  Kunar,  Logar,  Paktika,  Paktiya.  Wardak,  and  Ghazni  Provinces  over  a  period 
of  nine  month. 

Data  Source:  AMN  CIDNE 
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Descriptive  Statistics; 

Hot  Spots? 

*  Enemy-initiated  attacks  in  KUNAR  Province 


I1JW.I2 
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This  map  shows  a  small  number  of  incidents  along  the  province's  border  with  Pakistan,  in 
2011  there  was  a  significant  increase  of  indirect  fire  (IDF)  events  along  that  border  area. 
The  concentrations  of  IDF  events  occurred  around  Asadabadr  in  Darah-ye  Pech,  in  Bar 
Kunar  District,  andl  along  the  lines  of  communication  (LQC)  in  Ghaziabad  and  Nan  Districts. 


Similarly  to  IDF  events  in  Kumar  Province,  surface-to-air  fire  (SAFIRE)  events  have  increased 
slightly  from  109  to  130  events  for  a  period  of  S  month  2010  compared  to  the  2011.  The 
level  of  I  ED  explosions  in  Kunar  Province  has  remained  the  same  during  the  same  periods. 


Data  Sources  AMN  CIDNE 
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Descriptive  Statistics 

Survey  Trends 
Visibility  of  security  forces?  Perception  of  the 
population! 
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POLICE! 

The  results  from  ANQAR  Survey,  conducted  in  September  2011,  support  this  trend  by 
indicating  an  overall  lack  of  visibility  of  security  forces  in  Kunar  Province. 

AhlQAR:  Afghan  Nationwide  Quarterly  Assessment  Research 
Data  Source:  ANQAR  Survey,  as  of  September  2011. 
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ANA 

The  results  from  ANQAR  Survey,  conducted  in  September  2011,  support  this  trend  by 
indicating  an  overall  lack  of  visibility  of  security  forces  in  Kunar  Province. 

ANQAR:  Afghan  Nationwide  Quarterly  Assessment  Research 
Data  Source:  ANQAR  Survey,  as  of  Septem  ber  2011. 
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IED  Explosions  January-  Augusl 
^  IDOfi:  Orange 
■b  1009:  Purple 


lit}  Explosions  January  -  Augus! 
b  2010:  Green 
b  2011:  Blue 


Descriptive  Statistics ; 
Geospatial  Analysis 
Restrictions  of  Freedom  of  Movement 


luwia 
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The  rise  of  the  restrictions  of  freedom  of  movement  for  inhabitants  of  the  Kunar  Province  rs 
also  reflectedl  in  an  increase  of  IED  explosions  along  the  M3IR.  This  slide  shows  the  increase 
and  the  distribution  of  IED  explosions  in  Kunar  for  the  period  from  January  to  August  of  the 
last  four  years. 

Data  Source:  AM  N  CIDNE 
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B  Conclusions 

*  Important  situation  report  for  RC  FAST 

*  Statistically  support  to  impressions  of  the  PRT 
Kunar 

o  The  border  region  of  Kunar  Province,  especially  in 
the  vicinity  of  fhe  M5R  through  KUNAR  Province, 
has  shown  a  notable  increase  in  violence. 
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Conclusions. 

Several  factors  contribute  to  the  unique  and  segmented  violence  trends  in  RC  East  - 
disparate  and  highly  distributed  enemy  activity;  highly  restrictive  terrain;  and  the  proximity 
to  Pakistan.  Insurgent  activity  in  RCs  South  and  Southwest  tends  to  be  concentrated  in  key 
populated  areas.,  which  provides  better  opportunities  for  control. 

Recent  Combined  Team  and  Coalition  Force  successes  in  the  southern  portions  of 
Afghanistan  have  led  to  a  steady  and  progressive  decrease  in  levels  of  significant  activity 
(SIGACTs)  and  violence.  However,  the  situation  in  RC  east  is  different.  The  differences  are 
not  only  the  increasing  of  security  incidents,  also  in  particular  the  terrain  and  the  kinds  of 
attacks  are  different.  Friendly  forces  movements  are  only  possible  along  the  small  valleys, 
the  mountainous  area  offer  the  insurgents  various  hiding-places.  Further  on,  the  far  eastern 
pa  rt  of  the  region  is  one  of  the  poorest  areas  of  Afgha  nista  n  due  to  a  lack  of  infrastructure. 
Current  cross-border  fires  Into  Afghanistan  as  a  response  to  im  minent  threats  from 
insurgents  safe  havens  in  Kunar  and  Nuristan.  The  Pakistan  Army  reinforced  their  troops 
and  operates  with  25,000  troops  along  the  border  in  the  area  of  Nangarhar,  Nuristan  and 
Kunar  (N2K). 

Data  Source:  COMISAF  Morning  Update:  Combined  Joint  Intelligence  Operations  Center  - 
Afghanistan,  Pakistan  Border  Update,  as  of  09  October  2011. 
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Recommended  readings: 

•  Why  operations  assessments  fail  (Dr.  Jonathan  Schroden,  CNA) 

•Operations  Assessment  in  Afghanistan  is  broken  (Dr.  Stephen  Downes- Martin,  NWC) 
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A.2.7  Chernoff  Faces 


INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Operational  Assessment  PRT  KUNDUZ:  A  Case  Study  in  Visualization 

Faculty 

Name(s) 

Bundeswehr  Planning  Office,  Ottobrunn,  DEU 

Length 

30  minutes 

Learning 

Objectives 

Understand  that  visualization  methods  can  be  used  to  improve  the  situational 
awareness  of  the  decision-maker. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Recognize  similar  applications  where  an  analysis  by  OA  staff  could  support 
the  decision-making  process  by  improving  the  situational  awareness. 

Content 

Outline 

Case  Study:  Operational  Assessment  PRT  KUNDUZ. 

Materials 

None. 

Methods 

100%  Lecture  0%  Class  Exercises. 

%  Break  Outs. 

Exercise(s) 

None. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

Did  you  have  a  similar  case  in  the  past? 

Can  you  think  of  a  current  case  in  your  area  of  responsibility  where 
visualization  methods  could/do  improve  situational  awareness? 
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UTAN| 

Operational  Assessment 
PRTKUNDUZ  “ 


A  Case  Study  in  Visualization 

I?  Jan2QI3 


SI  Overview 

Situation:  Afghanistan  2007,  COM  ISAF  coordinates  the  NATO  efforts 
to  support  the  stabilization  arid  rebuilding  process  of  Afghanistan 
using  as  one  mean  the  Combined  Joined  Effect  Tasking  Order 
j  CJETO).  The  CJETO  prioritizes  sub  effects  |$Es]  for  each  PRT.  The 
large  number  of  SEs  assessed  complicates  the  task  to  provide  COM 
with  a  quick  overview  of  the  assessment. 

Decision:  Identify  suitable  visualization  methods/instruments. 

Objective:  Enable  COM  to  quickly  gain  an  overview  of  assessment. 

QA  Contribution:  Adapted  concept  of  Chemoff  Faces  to  visualize 
SEs  assessed  and  visualized  results  for  PRT  KUNDUZ  {use  case). 

Result  or  outcome:  intuitive  shared  (civii/miiitary)  situational  picture. 


In  order  to  describe  the  current  situation  in  the  24  districts  (PRT  KUNDUZ),  the  CJETO 
required  the  assessment  of  20  effects/sub  effects.  In  this  context,  the  Chernoff  Faces 
(Afghan  Glyphs)  were  intended  as  an  instrument  for  complexity  reduction. 
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Situation 


*  Situational  awareness  is  based  on  the 
assessment  of  a  large  number  of  sub  effects 

*  An  intuitive  shared  holistic  situational  picture, 
which  merges  civil  and  military  assessment 
and  allows  the  commander  to  get  a  quick 
overview  is  missing 


I™  Approach 

*  Identify  suitable  visualization 
mefhods/instrumenfs  to  enable  COM  to 

quickly  gain  an  overview  of  the  assessment 

*  Structure  the  assessment  with  the  the  high 
number  of  sub  effects  to  suitable 
parameters 

*  Implement  an  automated  tool  to  generate 
the  visualization  from  the  existing  raw  data 


a 
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Bl  OA  Contribution 


*  Identifying  Chernoff  Faces  as  suitable 
method 

*  Defining  together  with  the  COM  the 

parameters  to  be  represented  in  the 

Chernoff  Faces 

*  Building  the  tool  to  automate  the 
visualization 


Chernoff  Faces  (11  parameters) 


■  These  parameters  change  depending  on  the  value  of 
the  data  for  the  visualized  dimension. 

*  It  is  a  challenge  lo  rind  an  intuitive  mapping  of  ihe  data 
sets'  dimensions  to  the  parameters. 


Chernoff  Faces  were  invented  by  Herman  Chernoff,  an  american  applied  mathematician, 
statistician  and  physicist.  Their  purpose  is  to  visualize  multivariate  data  (see  slide  IS). 
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Parameters  to  be  Represented 


Hat;  Efficient  Governance 
Head:  Resources  (Water/Nutrition) 

Symmetry  of  Ears:  Stability 
Eyebrows;  Medical  Support 
Eye  Size:  Safety 
Pupil:  Electricity 
Nose:  Infrastructure 
Mouth;  Education 
Lacking  parts: 

No  current  (<  4  months)  information 


? 


Because  of  the  varying  perceived  importance  of  the  features,  the  mapping  (e.g.  safety  to 
eye  size)  should  be  carefully  chosen. 


Visualization  Tool 
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The  blue  box  (top  right)  contains  the  mapping  of  relevant  variables  to  the  features  of  the 
faces  (see  previous  slide).  The  situation  in  each  of  the  24  districts  (PRT  KUNDUZ)  - 
described  considering  eight  distinct  variables -is  visualized  by  a  Chernoff  Face. 
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Education? 
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Electricity,  Nutrition? 


Insights 


Security  situation 
around  KDI  unstable! 


■  Where  Is  a  lack  of  electricity 
or  water  and  nutrition? 

*  What  a  re  the  most  u  n*ta  ble 
districts? 


Information?  | 


Summary 


*  Visualisation  Method  identified  by  OA  and 
identification  of  relevant  parameters 
together  with  the  COM  provides  an  Intuitive 
shared  (civil/military)  situational  picture. 

*  To  best  utilize  your  OAs  to  aid  in  situational 
awareness  and  decision  making,  they  need 
access  to  you,  the  decision  maker* 

*  The  OA  brings  a  variety  of  tools  like 
visualisation,  optimization,  simulation, 
statistics,  and  assessment  skills  to  help  you 
with  evidence  based  decision  making. 
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Deriving  PRT’ s  Goal  System 

- MDG  2003 


CJETO 

Afghanistan 

Compacl 

r 

London 

T 

(02/2006) 

T 

(l)-ANDS 


Goal  System 
PUT  KDZ 


w 

i 

—Afghanistan 

Concept 

Gundesregierung 

(09/2007) 


1 

PDP 


ftrtDG  Wllfl-rw.im  Oaals 

(IJ-AWD5  [Interim)  Alflh&R  N^lionai  Dfrvelapffltftl  &lra'efly 
pdp  Frflvlniclgi  Development  Frojfcif 
CJETO  CG  Tibined  .  oinl  L^e-cHcskir-p  Order 


This  slide  indicates  the  differenct  elements  influencing  the  goal  setting  process. 


Lines  of  Operation  (LOP) 
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EBAO  =  Effects-Based  Approach  to  Operations 


Overview  CJETO 


LOP 


□E 


SE 


Explanation 


1  ■"  ?'  aA 


Priorily  Assess 


Supporting  Effects :  43 

Prioritised:  10 

Additional  report:  S 

National  addition:  2 

Tolal  #  qf  used  SEs :  20 

AOR  PRT  KUNDUZ:  24  Districbi^OSEs 

How  car  COM  get  a  quick  OVERVIEW? 


YES 
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^Ordered"  Supporting  Effects 
(CJETO) 
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Situational  Picture 
KUNDUZ  &  TAHKAR 
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This  slide  shows  the  old  approach  to  visualization. 
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S  What  are  Chernoff  Faces? 


•  The  Russian  mathematician  Chernoff  invented  -  most 
likely  coincidentally-a  visualization  technique  for 
multjvcriate  data.  Chernoff  visualized  his  do  to  sets  with 
facial  expressions, 

-  Muitivarate  data  means  that  there  are  many  dimensions 
within  data  sets. 

■  One  of  the  reasons  why  this  visualization  technique  works 
so  well  Is  because  human  beings  are  trained  to  inlerpret 
facial  expressions  fairly  quickly, 

Chernoff,  Herman  f  1973|:  The  use  of  faces  to  represent 
points  in  k-dimensional  space  graphically,  in:  Journal  of 
the  American  S to tisfical  Association.  63  (#342):  361-363. 

1-9 


Chernoff  Faces  (11  parameters) 


*  These  parameters  change  depending  on  the  value  of 
the  data  for  the  visualized  dimension, 

■  It  Is  a  challenge  to  find  an  intuitive  mapping  of  ihe  data 
sets'  dimensions  to  the  parameters. 

i? 
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The  Security  Situation 
(LOP  Security) 

*  Stability  is  visualized  by  the  ears  and  their  symmetry. 
Symmetric  ears  show  a  stable  situation  and  asymmetric 
ears  show  that  the  situation  is  highly  unstable. 

*  The  eye  size  visualizes  the  threat  potential.  A  high 
threat  potential  causes  large  eyes  indicating  potential 
fear.  Small  eyes  show  that  there  is  now  current  danger 
or  threat. 
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The  Civil  Situation  (1/3) 
(LOPs  Reconstruction/Governance) 


*  The  efficiency  of  the  local  administration  on  provincial 
and  district  level  is  visualized  with  o  hat.  A  small  hat 
means  that  the  administration  is  not  efficient.  A  large, 
well  formed  hat  means  that  there  exists  an  efficient 
administration  that  works  for  the  benefit  of  its 
populace, 


*  The  size  of  the  head  shows  the  supply  situation  with 
water  and  food.  A  big  round  head  has  abundant 
resources  whereas  a  small  and  tiny  head  still  suffers 
from  a  lack  of  potable  water  and  a  lack  of  nutrition 
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The  Civil  Situation  (2/3) 

( L O Ps  Recon s true ti on/G o ve mane e ) 


*  The  pupils  indicate  how  much  energy  Is  supplied  to  a 
district.  If  there  is  no  efficient  power  supply  the  pupils 
are  wide  open  to  indicate  that  it  is  dark  and  the  pupils 
adapt  appropriately.  Small  pupils  show  that  there  is 
enough  light  indicating  an  efficient  power  supply. 


*  The  eyebrows  indicate  how  well  the  health  provision 
functions.  Raised  eyebrows  indicate  a  happy  face 
meaning  that  the  healthcare  system  works  efficiently. 
The  angry  looking  face  shows  that  there  is  no  efficient 
health  care  provision  annoying  the  populace. 
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The  Civil  Situation  (3/3) 

( L  O  Ps  Recon  s  tructi  on/G  o vernanc  e ) 


*  The  length  of  the  nose  indicates  the  state  of  the 

infrastructure.  A  very  short  nose  can  be  associated  with 
a  lack  of  paved  roads.  A  long  nose  shows  that  the 
infrastructure  is  extended  and  functional  between 
district  centres  and  main  villages. 


*  The  mouth  shows  how  well  the  educational  system 
functions.  Male  (right  part  of  the  mouth)  and  female 
(left  part  of  the  mouth).  A  big  smile  means  that  the 
educational  system,  including  schools  and  vocational 
skill  training,  is  fully  functional.  A  sad  expression  means 
that  the  education  is  not  working  efficiently. 
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Matching  Visualization  and  SEs 


vi  suai  lotion 
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Supporting  Effects 
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Resulting  value:  High,  Medium,  Low  (simple  if-statements) 


2A 


Transition  to  Afghan  Glyph 


This,  figure  shows  that  we  have  no  current 
information  on  the  administration's 
efficiency  and  about  the  status  of  power 
supply. 

It  also  shows  that  the  infrastructure  and  the 
educational  syslem  have  medium  values. 
The  security  situation  is  mostly  eolm  and 
more  or  less  stable, 
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Use  Case: 

Campaign  Assessment  Tool 


*  Intuitive  shared  situational  picture* 

*  Merge  civil  and  military  situational  assessment, 

*  S  ta  nd  a  rd  ize  visua  I  ization  of  d  if f erent  da  ta. 

■  Integrate  system  down  to  raw  data. 
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Situational  Picture 
KUNDUZ  &  TAHKAR 
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Goal  achievement  on  district- level 
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Situational  Picture 
KUNDUZ  &  TAHKAR 


Goal  achievement  on  province-level 


Situational  Picture 
KUNDUZ  &  TAHKAR 
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interpret  the  pieiure; 

-  Where  h  g  lack  of  electricity  or  water 
and  nutrition? 

■  w  h  a  i  a  re  th  e  most  unstable  tiis trfcts? 
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Security  situation 
around  KDZ  unstable! 


Inform  o  lion? 


Htf  RT.iyrrf  •f-tjj.tyna  r-:.> 

Hn-:d.  F<+kv:il  | I mt.' h-Vi krii-iT< | 

SrfTnrmy™  Em  llni^b",' 

wdfcc i  5-.co>i 
rtpfc  BMtbffl y 
6t*rf  orion 

Lnr*ln|j 

IHa  nuiH-J  |r  4  rrc-rrtui  1 1 J  ■:■■  n  1 1  iwi 


sfigh 


Electricity.  Nutrition? 


Situational  Picture 
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El  District  Profile  Folder 
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A.3  STRATEGIC  PLANNING 
A.3.1  Strategic  Analysis 


INSTRUCTIONAL  MODULE  SUMMARY 

Title 

From  Concept  to  Operational  Method  for  Domestic  Operations,  The  Impact 
of  Strategic  Analysis:  Canada  Command  Effects-Based  Approach  to 
Operations 

Faculty 

Name(s) 

Brad  Gladman,  DRDC  CORA,  Ottawa  ON,  Canada 

Length 

60  -  90  minutes 

Learning 

Objectives 

Understand  the  value  of  the  distinct  capability  of  strategic  analysis. 

Appreciate  the  relationship  of  strategic  analysis  to  competent  operational 
planning  in  achieving  the  Commander’s  intent. 

Appreciate  the  need  for  an  appropriate  reporting  relationship,  and  other 
essential  requirements,  in  order  to  realize  the  potential  of  strategic  analysis 
in  decision-support. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Understand  the  value  of  the  distinct  capability  of  strategic  analysis  in 
decision-support. 

Understand  how  strategic  analysis  can  complement  competent  operational 
planning  in  the  development  of  a  clear  operational  method,  which  in  this  case 
was  used  to  change  other  government  department  thinking  about  the 
‘effects’  the  military  can  generate  in  various  crises  as  part  of  a  Whole  of 
Government  approach. 

Understand  what  strategic  analysis  requires  in  order  to  be  successful  in  any 
organization  it  supports. 

Content 

Outline 

Case  Study:  Canada  Command  Effects-Based  Approach  to  Operations. 

Vague  concept  to  operational  method. 

Concept  tailored  to  domestic/continental  operating  environment  as  part  of  a 
Whole  of  Government  approach. 

Summary  of  strategic  analysis  requirements  for  success  in  any  organization. 

Materials 

List  Required  Pre-Readings: 

Gladman,  B.  and  Archambault,  P.,  “An  Effects  Based  Approach  to 

Operations  in  the  Domestic  and  Continental  Operating  Environment:  A  Case 
for  Pragmatism”,  Ottawa:  DRDC  CORA  Technical  Memorandum  2008-033, 
2009. 
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Methods 

50%  Lecture  0%  Class  Exercises. 

_50%  Break  Outs. 

Exercise(s) 

Break  Outs 

X  Yes  No. 

#  5 _ participants  in  each  break  out  group. 

Discussion 

Questions 

Does  your  organization  have  an  embedded  strategic  analysis  capability? 

In  your  view,  does  it  require  such  a  capability? 

In  your  experience,  in  what  instances  would  your  organization  have 
benefited  from  strategic  analysis  for  decision-support? 

How  should  this  capability  be  structured  and  how  should  it  report  to  the 
senior  level  of  your  organization? 

If  this  capability  did  not  report  to  the  highest  level  of  your  organization,  was 
its  impact  impaired?  If  so,  how? 

M  EBB 

From  Concept  to  an  Operational 
Method  for  Domestic  Operations 


The  Impact  of  Strategic  Analysis: 
Canada  Command  Effects  Based 
Approach  to  Operations 


The  purpose  of  this  presentation  is  to  understand  the  value  of  the  distinct  capability  of 
strategic  analysis;  appreciate  the  relationship  of  strategic  analysis  to  competent 
operational  planning  in  achieving  the  Commander's  intent;  and  appreciate  the  need  for 
an  appropriate  reporting  relationship,  and  other  essential  requirements,  in  order  to 
realize  the  potential  of  strategic  analysis  in  decision-support. 


Strategic  Analysis  is  the  study  of  the  use  of  instruments  of  national  power  to  achieve 
strategic  objectives,  with  an  emphasis  on  understanding  military  power.  Using  methods 
from  Strategic  Studies,  Social  Sciences,  History  and  data  from  open  and  classified 
sources,  Strategic  Analysts  develop  strategic  outlooks,  assessments,  case  studies, 
scenarios,  military  trend  analyses,  and  strategic  lessons  learned  to  support  and  inform 
strategic  decision-making  within  the  Canadian  Armed  Forces  and  the  Department  of 
National  Defence. 
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Overview 


Situation:  The  use  of  an  Effects -Based  Approach  to  Operations 
[EBAG)  became  part  of  the  Canada  Command  Concept  of 
Operations  in  2006. 

Decision:  Develop  an  understanding  of  the  concept,  endorse 
and  apply  it  for  use  in  the  domestic  and  continental  operating 
environment. 

Objective:  Inform  the  Commander  about  whether  this  concept 
adds  anything  of  value  to  the  current  Operational  Planning 
Process  (GPPL  and  suggest  a  means  to  apply  If  effectively  for 
domestic  and  continental  operations. 

OA  Contribution:  The  Strategic  Analysts  dispelled  many  of  the 
‘fanciful’  notions  sunrounding  the  concept  and  placed  it  in  the 
context  of  a  'whole  of  government1  approach  to  operations.  A 
means  was  developed  to  use  the  concept  to  influence  Other 
Government  Department  [OGD]  thinking  about  the  military  role 
in  domestic  crisis  response. 


Result  or  outcome:  The  pragmatic  use  of  EBAG  in  the 
development  of  domestic  contingency  plans  and  in  strategic 
discussions  with  OGD  officials. 
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In  2005  the  Canadian  Chief  of  the  Defence  Staff  initiated  an  intensive  transformation 
effort  built  around  a  series  of  guiding  principles  designed  to  reorganize  the  command 
and  control  of  the  Canadian  Armed  Forces  to  make  it  more  responsive  and  effective,  and 
thereby  more  relevant.  One  of  the  recommendations  made  was  the  establishment  of  an 
integrated  operational  level  command  responsible  for  domestic  operations,  Canada 
Command. 


As  this  command  established  itself,  its  first  concept  of  operations  indicated  it  would 
adopt  an  effects-based  approach  to  operations. 


The  strategic  analyst's  task  was  to  look  pragmatically  at  the  concept  and  find  a  way  to 
apply  it  effectively  for  operations  in  the  Command's  area  of  responsibility.  Iln  so  doing, 
the  analyst  helped  to  move  the  command  towards  an  operational  method  that  enhanced 
strategic  discussions  with  other  government  department  officials. 


organization 
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B  Situation 

*  In  2006,  the  Canadian  domestic  Command's 
concept  of  operations  indicated  an  effects-based 
approach  to  operations  would  be  adopted,  yet  it 
was  quite  clear  that  most  staff  did  not  understand 
what  that  meant  in  the  context  of  a  Command 
conducting  domestic  operations  in  the  Canada 
Command  Area  of  responsibility  (AOR). 

*  While  the  Commander's  direction  was  accepted, 
few  had  any  idea  what  this  concept  added,  if 
anything.,  to  the  current  OPR; 


Military  operations  in  Canada  and  its  approaches  are,  far  the  most  part,  in  support  of 
other  government  departments.  Excepting  defence  of  Canada  and  other  purely  defence 
missions,  the  military  acts  only  in  support  of  Provincial  or  Federal  authorities  under 
very  specific  legal  authorities. 


At  times  before  the  establishment  of  Canada  Command,  other  government 
departments  and  Provincial  authorities  viewed  the  military  as  a  warehouse  from  which 
to  draw  equipment  and  personnel  for  various  purposes.  In  one  instance  in  1999  the 
mayor  of  Toronto  asked  for  military  assistance  to  shovel  snow  following  a  blizzard. 
Because  of  the  direct  reporting  relationship  of  the  analysts  within  the  command,  the 
strategic  analyst  knew  the  Commander  wanted  the  way  OGDs  Mewed  the  military  to 
change.  The  strategic  analyst  saw  the  EBAO  concept  as  a  possible  means  to  that  end. 
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Analyst  Approach 

*  Concept  of  Operations:  the  use  of  EBAO  would  be 
central  to  Commander  Canada  Command’s 
approach  to  operations. 

*  Hypothesis:  it  is  possible  to  use  this  basic  concept 
stripped  of  fanciful  notions,  to  change  the  way 
civilian  authorities  view  the  new  Command  and  its 
role  in  domestic  operations  in  the  current  security 
environment. 

*  Analysis  Approach:  tailor  the  concept  to  the  mission 
and  the  domestic  environment  and  encourage  the 
Commander  and  staff  to  develop  plans  and  speak 
to  OGD  in  terms  of  ’effects  delivered’  instead  of  'kit 
provided'. 


In  order  to  apply  this  concept  towards  that  end,  the  strategic  analyst  examined  the 
effects-based  operations  concept  and  tried  to  tailor  it  to  the  realities  of  military 
operations  in  the  domestic  operating  environment.  After  a  great  deal  of  analysis  of  Allied 
approaches  to  this  concept,  it  was  determined  that  the  best  way  forward  initially  was  to 
suggest  that  the  Commander  and  staff  develop  plans  and  speak  to  OGD  officials  in  terms 
of  military  'effects  delivered’  instead  of  'kit  provided'. 


The  suggested  approach  was  quite  simple,  and  quite  different  to  the  approach  typically 
suggested  by  effects-based  operations  concept  developers.  Its  aim  was  to  change  how 
the  military's  role  in  domestic  contingencies  was  viewed  by  civilian  authorities. 
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ES3  Strategic  Analyst  Contributions 
a  (1  of  2) 

•  Tailoring  the  Concept 

a  Dispel  some  of  the  fanciful  notions  surrounding 
the  concept  including  the  assumption  of  near 
certainty  and  mechanistic  approaches  which 
supplant  the  operational  art. 

*  Operational  Method 

o  If  placed  in  its  proper  context,  the  concept 
would  assist  Commander  Canada  Command  to 
educate  civilian  authorities  about  the  kind  of 
effects  the  command  can  deliver,  and  thus  help 
develop  a  more  collaborative  approach  to 
operations. 


Before  doing  so,  it  was  necessary  to  examine  the  concept  pragmatically  and  to  strip  it  of 
its  fanciful  notions  (including  its  mechanistic  approach  to  operations  and  the  ability  to 
develop  and  maintain  near  certainty). 

If  placed  in  an  appropriate  context,  the  concept  could  assist  Commander  Canada 
Command  to  educate  civilian  authorities  about  the  kind  of  effects  the  command  could 
deliver,  something  which  would  foster  a  collaborative  approach  to  operations. 
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Egg  Strategic  Analyst  Contributions 
■■  (2  of  2) 

*  Whole  of  Government  Approach 

*  Suggest  an  essential  framework  in  which  this 
concept  can  be  applied  usefully  to  the 
domestic  and  continental  operating 
environment. 

*  Doing  so  placed  Canada  Command 
operations  and  their  desired  effects  in  the 
broader  context  of  national  security  and  the 
strategic  or  ‘Whole  of  Government’ 
framework  required  therein. 


In  essence,  the  analyst  argued,  an  effects-based  approach  allows  the  military  to  work 
within  a  Whole  of  Government  framework,  while  providing  a  language  that  reflects  the 
military's  unique  capabilities.  And  that  uniqueness  must  be  protected  as  a  matter  of 
course.  It  is  necessary  for  the  CAF  not  to  civilianize  its  planning  processes. 


Furthermore,  the  strategic  analyst  argued,  the  military  should  not  drive,  or  be  perceived 
to  be  driving,  the  Whole  of  Government  approach  within  the  domestic  context  since  in 
most  cases  the  military  acts  in  support  of  a  lead  department  or  Provincial  authority. 
Being  the  400  pound  gorilla  in  the  room  during  planning  meetings  with  civilian 
authorities  would  not  produce  a  collaborative  approach. 


For  domestic  operations,  the  strategic  analyst  concluded!  that  EBAO  doctrine  should  not 
be  made  more  complicated  than  need  be.  And  really  all  it  needs  to  be  is  codified  as  a 
pragmatic  iteration  of  best  practices  identified  by  operators  based  on  simple  guiding 
principles. 


The  analysis  of  the  concept  led  to  a  general  approach  to  operations  that  highlighted  the 
military's  unique  capabilities  and  'effects'  iit  could  deliver  during  a  crisis  response.  This 
differed  markedly  with  the  general  perception  that  the  military  was  something  to  call 
when  you  need  x  number  of  cots  and  some  trucks  and  food. 
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Strategic  Analysis 
Requirements  for  Success 


•  In  this  instance,  a  direct  reporting  relationship  with  the 
Commander  was  essential  to  understand  his 
perspectives  on  domestic  operations  and  the  need  to 
alter  how  civilian  authorities  viewed  his  command. 

•  Strategic  Analysis  is  a  unique  form  of  operational 
research  and  analysis,  and  requires  unfiltered  access 
to  the  top  level  of  whatever  organization  it  is 
supporting.  Without  this  access,  the  ability  to 
comprehend  fully  the  true  strategic  issues  is  Impaired. 

•  The  result  of  reporting  to  the  Command  level  was  the 
ability  to  assist  in  developing  an  operational  method 
that  exceeded  the  commander's  expectations. 
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In  this  instance,  the  success  of  this  project  was  due  in  large  part  to  the  strategic  analyst 
having  a  direct  reporting  relationship  with  the  Commander,  something  essential  to 
understand  his  perspectives  on  domestic  operations  and  his  need  to  alter  how  civilian 
authorities  viewed  his  command  and  the  military  in  general. 


Strategic  analysis  is  a  unique  form  of  operational  research  and  analysis,  and  requires 
unfiltered  access  to  the  top  level  of  whatever  organization  it  iis  supporting.  Without  this 
access,  the  ability  to  comprehend  fully  the  true  strategic  issues  is  impaired.  In  this 
instance,  that  access  produced  an  operational  method  that  exceeded  the  Commander's 
expectations. 
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An  Example  -  Op  PONTOON 


•  The  first  example  of  the  revised  effects- based 
approach  to  operations  took  place  in  the  spring 
and  early  summer  of  2007,  when  Canada 
Command  planned  for  lia  scalable  CF  response  to 
support  the  civil  authorities  in  [British  Columbia]  in 
their  flood  mitigation  and  response  effort",  in  broad 
terms,  this  operation  was  a  textbook  example  of  an 
‘effects-based  operation'. 


•  As  a  result  of  intense  planning  discussions,  the  British 
Columbia  Provincial  authorities  were  encouraged 
to  not  simply  ask  for  personnel  and  equipment. 
Instead,  the  BC  authorities  would  request  support  in 
the  form  of  specific  effects  desired  when  the 
situation  reached  a  point  where  they  were 
overwhelmed. 


A  specific  example:  Op  Pontoon.  The  Canadian  Forces  response  to  the  potential  flooding 
of  the  Fraser  River  Valley  was,  in  broad  terms,  a  textbook  example  of  an  effects-based 
operation. 


In  preparing  for  the  operation,  the  strategic  intent  of  both  the  British  Columbia  provincial 
authorities  and  the  senior  leadership  of  DND/CF,  and  the  expected  nature  of  the  CF 
participation,  was  clearly  spelled-out  and  nested  within  the  operational  orders.  Thus,  the 
operational  level  planning  was  synchronised  with  the  desired  strategic  end-state,  as  was 
the  tactical  planning. 
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The  Fraser  River  Valley 
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This  is  an  image  of  the  Fraser  River  Valley,  along  which  are  a  number  of  large  urban 
centres  including  one  of  the  largest  cities  in  Canada,  Vancouver.  The  potential  impact  of 
large-scale  flooding  along  this  river  is  easy  to  envisage,  as  is  the  potential  for 
overwhelming  local  and  provincial  authorities. 
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H  Op  PONTOON 

*  The  effects  the  CAF  would  achieve  were  arrived  at 
through  a  Whole  of  Government  lens  and 
approach. 

*  Numerous  government  departments  and  agencies 
at  all  levels,  from  local  municipal  authorities  through 
to  Public  Safety  Canada,  and  from  the  Canadian 
Coast  Guard  through  to  the  CAF,  contributed  to  a 
clear  understanding  of  the  potential  challenges 
and  what  each  could  add  to  the  combined  effort. 

*  in  this  instance,  the  suggested  approach  of 
speaking  to  OGD  officials  in  terms  of  military  'effects 
delivered 1  was  a  success. 

10 


Various  government  departments  and  agencies  from  the  local  municipal  authorities 
through  to  federal  authorities  including  Public  Safety  Canada,  the  Canadian  Coast  Guard 
and  the  CAF  contributed  to  a  clear  understanding  of  the  situation  and  what  each  could 
add  to  the  combined  'whole  of  government'  effort.  It  marked  a  substantial  change  in 
approach  and  how  the  other  departments  viewed  Canada  Command  and  what  it  could 
provide. 


In  this  instance,  the  flooding  was  not  as  damaging  as  anticipated,  and  yet  Canada 
Command  planners  had  prepared  a  phased  response  tied  to  a  series  of  water  level 
indicators  developed  in  collaboration  with  civilian  authorities.  The  military  was  in  a 
position  to  provide  its  unique  effects  in  a  timely  fashion  if  required,  and  the  strategic 
relationship  between  Canada  Command  and  its  supported  partners  was  clearly 
understood  and  appreciated. 


The  main  contribution  of  the  strategic  analyst  in  this  effort  was  to  help  shape  the 
Command's  approach  to  such  operations,  something  which  enhanced  operational 
planning  and  in  changing  how  civilian  authorities  regarded  the  military.  The  success  of 
this  effort  and  the  analyst's  role  had  certain  requirements. 


organization 
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•  The  ability  of  strategic  analysis  to  fulfill  its 
potential  is  contingent  on  its  reporting 
relationship  to  senior  leadership/decision 
makers. 

•  If  properly  located,  it  Is  possible  to  get  a  first¬ 
hand  understanding  of  the  strategic  issues 
requiring  analysis,  as  well  as  an  understanding 
of  the  Commander's  intent  which  can  then  be 
informed  and  refined. 


First,  the  strategic  analyst  had  to  report  directly  to  senior  leadership.  That  relationship 
enabled  art  understanding  of  the  truly  strategic  issues  requiring  analysis,  as  well  as  art 
understanding  of  the  Commander's  intent  which  can  then  be  informed  and  refined.  The 
more  levels  in  the  way  and  the  more  filters  through  which  the  analyst  has  to  speak  can 
seriously  degrade  the  ability  to  provide  effective  strategic  analysis. 
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3  Summary  2/3 

■  In  this  instance,  unclear  and  vague  references 
to  Canada  Command’s  'Effects-iBased 
Approach  to  Operations'  were  refined  into  a 
useful  operational  method  with  the  assistance 
of  strategic  analysis. 

*  Strategic  Analysis  did  not  replace  competent 
operational  level  planning,  but  rather  helped  to 
refine  an  understanding  of  the  concept  and 
Ihow  it  could  be  used  in  domestic  operations  as 
part  of  a  Comprehensive  Approach. 


In  this  instance,  and  due  in  large  part  because  of  the  direct  reporting  relationship  and 
personal  credibility,  an  initial!  uncertainty  about  how  to  apply  an  Effects-Based  Approach 
to  Operations  from  the  Commander's  Concept  of  Operations  was  refined  with  the 
assistance  of  strategic  analysis  into  a  useful  operational  method. 


Strategic  Analysis  did  not  replace  competent  operational  level  planning,  but  rather 
enhanced  it. 
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3  Summary  3/3 

*  Once  stripped  of  ‘fanciful  notions' 
surrounding  the  concept  EBAO  became  a 
useful  tool  to  shape  OGD  perception  of  the 
Command  and  what  it  could  deliver. 

*  The  impact  of  strategic  analysis  in  this 
instance  was  considerable  and  lasting. 

*  The  report  documenting  the  research  and 
analysis  has  been  made  required  reading 
for  all  new  and  existing  Canada  Command 
planners  by  the  DCOS  [Plans  and  Strategy).1 

*  Strategic  Analysis  has  a  key  role  to  play  in 
CD&E. 


Once  stripped  of  the  more  'fanciful  notions'  surrounding  the  concept,  an  effects-based 
approach  to  operations  became  a  useful  tool  to  shape  the  perception  of  Canada 
Command  and  its  role  in  this  kind  of  domestic  operation. 


Because  of  the  close  working  relationship  between  strategic  analysis  and  the 
Commander,  the  effect  was  considerable  and  lasting.  The  report  documenting  the 
research  and  analysis  leading  to  the  recommended  operational  method  was  made 
required  reading  for  all  Canada  Command  planners. 


STO-TR-SAS-098 


ANNEX -151 


ANNEX  TO  SAS-098 


organization 


E  IS  Endnotes 

1  Brad  Gladman  and  Peter  Archambault  An  Effects 
Based  Approach  to  Operations  in  the  Domestic  and 
Continental  Operating  Environment:  A  Case  for 
Pragmatism  (Ottawa:  DRDC  CORA  Technical 
Memorandum  2008-033,  2009)* 
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Whither  the  Operational  Art? 
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Cracks  in  the  Foundation 


*  Some  proponents  of  EBAO  have  argued  its  revolutionary 
nature  and  discontinuous  departure  from  past 
operational  methods,  but  in  fact  the  principles 
underlying  the  concept  are  not  new.  The  best  military 
commanders  and  the  most  capable  statesmen  always 
have  focused  on  attaining  the  desired  end  state  and 
did  not  discount  the  human  dimension  of  conflict.  They 
routinely  mobilised  all  elements  of  national  power  to 
attain  the  desired  end-state, 

•  Whi letechnological  advances  h a v e i m pacted 
operations  across  the  spectrum  of  conflict,  they  have 
not  eliminated  many  of  the  problems  encountered  in 
operations,  or  so  revolutionised  the  approach  to  the 
military  art  that  past  experience  is  no  longer  relevant. 


IB 
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The  Assumption  of  Near-Certainty 


*  The  belief  in  certainty  in  future  military  operations 
has  resurrected  and  cloaked  in  a  new  lexicon  an 
old,  failed  strategy  from  the  Vietnam  war  in  which  it 
was  believed  that  military  actions  could  be 
precisely  applied  to  achieve  strategic  objectives  in 
a  predictable  process. 

•  In  conflict  or  operations  where  adversaries  or  the 
environment  strike  at  the  ability  to  maintain  this  near 
perfect  understanding  of  the  situation,  the  need  for 
an  operational  art  -  the  ability  of  commanders  to 
determine  the  correct  course  of  action  while 
shrouded  by  either  a  lack  of  information  or  by  too 
much  rapidly  changing  information  -  remains  as 
essential  for  success  as  it  always  has. 


The  Assumption  of  Near-Certainty 


•  When  considering  EBAO  tor  the  domestic  and 
continental  environment,  it  was  important  to  look 
pragmatically  at  the  concept  and  avoid  making  it 
into  more  than  it  is  or  should  be. 

*  Moreover,  shop-worn  cliches  about  its  revolutionary 
nature  were  dispensed  with,  along  with  the 
mechanistic  and  unrealistic  underpinnings  of  near 
certainty  and  the  ability  to  maintain  it  throughout 
an  operation.  Only  then  was  it  possible  to  address 
coherently  the  role  of  EBAO  in  a  domestic  context, 
assess  its  value  to  contingency  and  crisis  operations, 
and  recommend  an  appropriate  process  in  which  it 
should  function. 


30 


A.3.2  Peace  Support  and  Game  Theory 

(see  Application  Area  1  -  Section  A.2.4) 
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A.4  TRANSFORMATION 
A.4.1  Base  Realignment  and  Closure 


INSTRUCTIONAL  MODULE  SUMMARY 


Title 

U.S.  Army  Base  Realignment  and  Closure  (BRAC)  Using  Military  Value 
Analysis 

Faculty 

Name(s) 

Dr.  Lee  Ewing,  Naval  Postgraduate  School,  Monterey,  California,  USA 

Length 

20-30  minutes 

Learning 

Objectives 

Understand  the  use  of  operational  research  techniques  to  logically  assess 
both  the  qualitative  and  quantitative  aspects  of  value  analysis. 

Gain  an  appreciation  for  the  relationship  is  strategic  level  resource  decision¬ 
making  and  creating  analytical  objectives  to  support  that  decision. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Appreciate  the  ability  to  assess  alternatives  in  a  strategic  resource  decision 
both  qualitatively  and  quantitatively. 

Understand  the  contribution  of  decision  analysis  techniques  to  aid  in 
decision-making. 

Content 

Outline 

Case  Study:  BRAC  decision. 

Mapping  Value  to  alternatives  in  the  decision  process. 

Creating  two-dimensional  value  matrix. 

Creating  an  objective  function  for  multiple  optimization  problems. 

Materials 

List  Required  Pre-Readings: 

None. 

Methods 

80%  Lecture  20%  Class  discussion. 

%  Break  Outs. 

Exercise(s) 

None. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

In  your  experience,  where  has  value  mapping  been  used  to  aid  decision¬ 
making? 
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Army  Base  Realignment  and 
Closure  (BRAC)  2005  Military 
Value  Analysis 


A  Cose  Study  using  Decision  Analysis 

1MU4 


IB  Overview 


Situation:  WHAT  IS  BRAC?  A  process  the  Services  use  to  reduce 
infrastrucfure  end  operating  costs,  in  a  politically  sensitive 
environment 

Decision:  As  mandated  by  the  BRAC  law.  determine  the  military 
value  ot  an  Army  installation. 

Objective:  Develop  the  Army's  BRAC  position  which  provides  ihe 
maximum  military  value. 

OA  Contribution  Provided  an  ouditobler  repeatable,  and 
defendable  approach  that  supported  Army/DoD  decision  makers 
resulting  in  a  95%  Presidential  Commission  approval  rating  for  the 
Army  f vs.,  85%  overall) 

Result  or  outcome:  A  two  year  study  that  resulted  in  a  $5G  Billion 
program,  involving  US  Armed  Forces.,  millions  of  people.,  hundreds  at 
locations,  oversees  forces,.  soldiers  and  their  families 

I2JW.U  2 
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S  Approach 

■  The  Qualitative  Aspect 

o  Informed  by  36  senior  leader  interviews, 
numerous  staff  interactions,  and  an  extensive 
literature  review 

o  Consists  of  six  primary  competing  objectives  and 
40  supporting  attributes 

o  Forms  the  framework  for  the  quantitative  analysis 
•  The  Quantitative  Aspect 

o  Quantitative  functions  are  defined  by  the 
qualitative  attributes  which  directly  support 
previously  defined  objectives 

o  Subjective  nature  of  weighting  is  reduced 


Analysis  Objectives 
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S  OA  Contribution 

*  Supported  major  Army  decisions  to  enable 
dramatic  cost  savings, 

*  Army  BRAC  submissions  had  a  95%  Presidential 
Commission  approval  rate  (85%  overall). 

o  Result*  are  auditobie  and  repeatable  because  ttie  process  is 
well  documenled  and  not  □  "black  box” 
o  Analysis  is  accepted  because  it  is  qualitatively  and 
quantitatively  defendable 

*  OA  analysts  made  two  important  technical 
contributions 

o  Swing  Weight  Matrix 
o  £  Dimensional  value  measures 


BB  Summary 

*  Decision  Analysis  Is  appropriate  for  strategic 
and  operational  decisions  which  are 
characterized  by  competing  objectives 

*  Provides  vision  on  a  way  forward  for  difficult 
policy  issues 

*  Very  powerful  technique  to  determine  the 
objective  function  for  multiple  objective 
optimization  problems 
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Army  Installation  Military 
Value  Analysis 


*  We  used  four  major  decision  analysis  models  within  Army  BRAC 
■  We  mm  hi  ned  dec  isiou  a  nal vsis  technique  w  i  rh  opt  i  m  i  xa  rinn  tecliniqa^  to 
provide  a  ranking  of  instdlatLcmH  based.  cm  value,  a  portfolio  of  installations  based 
on  valine  and  cost,  a  ranking  of  options  based  on  value,  a  nd  a  portfolio  of  options 
bashed  on  value  and  ctml. 


Module 

IEAI 

(IriH-tallalmn  Evnl  iiuMmi  Ifludulej 

ODEM 

iQpbon  Devd  up  merit  end  Fwilunt  cti  ModiJc) 

Models 

MVI 

(MV-lnilullatloiwih 

MVP 

(MV-Portfolio* 

OVM 

(Option  Vhh»  Model) 

OPM 

(■Dp-lion  Portfolio 
Model) 

Products 

In:-;tnllj1i-nn 

Portfolio 

Scenprip 

Qpfcn 

Evsluaiion 

Debemiliifllion 

Value 

Evaluallon 

w-' 

loday's  Focus 


1 .  The  current  and  fut  u  re  m  issi  on  ca  pa  b  i  I  ities  and  the  im  p  a  ct  o  n  operat  i  ona  I  re  ad  rn  e  ss 
of  the  Department  of  Defense's  total  force,  including  im  pact  on  joint  warfighting, 
training,  and  readiness, 

2.  The  availability  and  condition  of  land,  facilities  and  associated  airspace  (including 
training  areas  suitable  for  maneuver  by  ground,  naval,  or  air  forces  throughout  a 
diversity  of  climate  and  terrain  areas  a  nd  staging  areas  for  the  use  of  the  Armed 
Forces  in  homeland  defense  missions)  at  both  existing  and  potential  receiving 
locations. 

3.  The  ability  to  accommodate  contingency,  mobilization,  and  future  requirements  at 
both  existing  and  potential  receiving  locations  to  support  operations  and  training. 

4.  The  cost  of  operations  and  the  manpower  implications. 
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Certified  data  and  many  studies 

EBB  required  to  support  the  MV  decision 

E23  analysis 

Anribute 

Supporting  Office 

Soil  Resiliency  {1  \  ) 

.Army  Environmental  Center  (A EC] 

Deployment!^  IS) 

Military  Suriaue  [J^phiyui  ltiL  imd. 

Distribution  Comroaiui,  Transportation 
Engineer  ng  Agency  (TEA) 

Accessibility  ( 17} 

Center  few  Amy  Analysis  (CAA) 

Connectivity  <1 81 

Arrny  G-6 

Work  Porte  Availability  (31) 

OfTiOft  ttf  Ettmornii-  and  Manpower  Anal  vhir 
at  West  Point  (OP  VIA) 

Envimnmjefital  Elasticity  (3BJ 

Army  Environmental  Policy  Institute  (AEP1) 

Urban  Sprawl  (JQ) 

Cnnstruclirm  I'lnuincm  njt  k^s^arih 

L.;sbciraccT>‘  (CIH-K 1  ) 

Critical  InfrasmustiiDe  Proximity (40) 

CAA 

Other  Papers;  Ornate  <S  Terrain  Analysis 
Staging  Arvo  Analysts 

TAHS  and  \CSlM  (Ci]S  Group} 

Military  Value  Calculation 


Process  steps  include 

□  Select  Attribute!  (A)  1  c 

o  Select  Weight  (w,) 

To  calculate  MV 

\felue 

o  Find  the  score  for  an  attribute 

□  Convert  the  score  to  c  value 
o  Sum  the  weight  of  each 

attribute  multiplied  by  the 
value  for  each  attribute 

o  MVi  =  Z,  w,  V(Ai ),  max  MV  of  10 


One  value  function  for 
each  attribute 


Score 

(Inslallation'a  inputs 
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CSS3  Two  Dimensional 
1333  Constructed  Measures 


TOTAL  HVY  MVR  AREA  (1000s  acres) 

Largest 

Contiguous  Area 
{1000s  acres) 

<-10 

>10  and 
<=50 

>50  and 
<=  100 

>100 

<  =  10 

0.1 

0.2 

1.3 

2.0 

>10  and  <  =  50 

X 

32 

4.2 

5.1 

>50  and  <  =  100 

X 

X 

6.1 

7.6 

>100 

X 

X 

X 

10.0 

«  Captures  complex  dependencies  relationships 
*  Still  allows  os  to  use  additive  value  model 


Benefits 

Captures  the  value  dependence  between  several  attributes 
Captures  qualitative  and  quantitative  measures  in  one  attribute 
Matrix  offers  visual  perspective  and  SME's  more  comfortable  with  value  assessments 
Results  in  fewer  attributes  in  MOD  A 
Limitations 

Difficult  to  develop  and  harder  to  assess  than  natural  scales 
What  measures  (bins)  are  appropriate? 

Must  be  consistent  when  evaluating  SME  preferences 
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Swing  Weight  Matrix:  BRAC  Attributes 

Increasing  ability 
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Visual  Approach  helps  resolve  some  of  the  trtutiikmtil  issues  with 

assessment  of  swing  lueigkte 

Driven  by  the  need  to  explain  rationale  for  the  weights  to  auditors  a  nd  BRAC 
Commission 

Difficult  to  gain  consensus  with  a  "group"  of  decision  makers 
Subjectivity  issues 
Inherent  preferences 

Considered  existing  assessment  techniques 
Swing  weights 
Pair-wise  comparisons 


Benefits 

Visually  depicts  importance  and  variation  dimensions  of  the  problem 

Provides  framework  for  consistent  swing  weight  assessments 

Provides  auditable  weighting  rationale  that  can  be  explained  to  senior  leadership 

Limitations:  Requires  some  professional  judgment 
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jjjjjj  Military  Value  -  Portfolio  (MVP) 

-  MVi  provides  the  Inputs 
*  Mi  litary  Va  lue  P  ortfol  io 

hscallxtin 

Military  Value 

i. 

Ranking 

rtV| 

...  Goal:  Recommend  a  set  of  instalations  (an 

Army  pOllfoJpOj  required  t-0  support  Army  needs 

w ■'  iio  it oin _a  nirg  ‘ho  max  p-urn  Mi  iiary  Value 

It  Reauired 

►  Install  □  tions  not  in  Portfolio 

Capabilities  and 

.  Storting  paint  tor  BRAC  actions, 

o  Af  risk  from  a  medei  perspective:  TABS  odds 

military  judgment  to  determine  final  outcome 

Portfolio 

*  Mi  lilary  Value  is  one  oipect  TABS  works, 

scenarios  to  determine  feasibility  of  BRAC 

ED. 

la. 

actions 

1  . 

fii-  ..  BRAC 

\  Actions 
w.  J 

Benefits 

Captures  the  value  dependence  between  several  attributes 
Captures  qualitative  and  quantitative  measures  in  one  attribute 
Matrix  offers  visual  perspective  and  SMB's  more  comfortable  with  value  assessments 
Results  in  fewer  attributes  in  MOD  A 
Limitations 

Difficult  to  develop  and  harder  to  assess  than  natural  scales 
What  measures  (bins)  are  appropriate? 

Must  be  consistent  when  evaluating  SME  preferences 
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Value  at  Different  Risk  Levels 

(m)  • 


TOM  - 

KJW 

50  DO  - 

s 

40  M  - 

E 

Q. 

MM 

o 

20  DO 

10DQ  - 

Change  in  Valua 
/Change  In  $ 

Value 

Dollars  flB'i 

Min 

f-l'A 

920 

$4.33 

LOW 

2D  73 

4  sag 

$6.10 

MID 

261 

&497 

$  9  sa 

NI5N  | 

6B 

5707 

SI  3.66 

4  5  6  7  5  9  10  11  12  13  14  15 

lmplemefiia1ionCo5l5.  inclining  Savings  ($B) 


Benefits 

Captures  the  value  dependence  between  several  attributes 
Captures  qualitative  and  quantitative  measures  in  one  attribute 
Matrix  offers  visual  perspective  and  SME”s  more  comfortable  with  value  assessments 
Results  in  fewer  attributes  in  MOD  A 
Limitations 

Difficult  to  develop  and  harder  to  assess  than  natural  scales 
What  measures  (bins)  are  appropriate? 

Must  be  consistent  when  evaluating  SME  preferences 
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Decision  Analysis  Best  Practices 

*  Perform  senior  leader  stakeholder  analysis 

*  Eliminate  unnecessary  analysis  restrictions 
»  Use  good  decision  analysis  practice 

o  Minimized  the  number  of  measures 

o  Swing  weight  matrix  is  very  useful  to  develop 
and  defend  weights 

o  Two-dimensional  value  measures  solved 
dependence  issues  and  were  easier  to  obtain 


Be  prepared.  The  Army  analysis  team  had  significant  experience  in  the  problem  domain 
(Army  stationing  and  BRAC)  and  extensive  experience  performing  operations  research  and 
decision  analysis  for  senior  leaders. 

Perform  senior  leader  stakeholder  analysis.  The  Army  senior  leader  interviews  provided 
important  perspectives.,  helped  identify  transformation  opportunities,  and  enabled  later 
access  to  key  subject  matter  experts  to  help  develop  models. 

Eliminate  unnecessary  constraints.  Unlike  BRAG  1995,  we  removed  the  installation 
category  constraint.  This  allowed  us  to  consider  the  military  value  of  an  installation  for  any 
Army  mission.  This  increased  the  solution  space  by  allowing  possible  alternatives  that 
moved  missions  between  installation  categories. 

Use  good  decision  analysis  practice.  Use  an  appropriate  technique.  First,  we  minimized 
the  number  of  attributes.  This  meant  we  could  obtain  high  quality  data  for  fewer  high 
quality  measures.  Second,  the  two-dimensional  measures  addressed  the  value 
dependence  problem  and  helped  gain  senior  leader  support.  Finally,  the  Swing  Weight 
Matrix  was  very  useful  to  assess,  explain,  and  defend  weights. 

Ignore  bad  advice.  We  received  three  recommendations  that  we  did  not  follow.  First, 
during  our  initial  assessment,  some  individuals  who  had  worked  BRAC  1995  warned  us  not 
to  change  the  successful  BRAC  1995  process.  We  viewed  BRAC  2005  as  fundamentally 
different  due  to  the  emphasis  on  transformation  and  joint  warfighting.  Secondly,  some 
thought  it  was  a  very  bad  idea  to  interview  senior  leaders.  This  was  easy  advice  to  ignore. 
Finally,  some  of  our  friends  warned  us  not  to  '"''waste  our  time  working  on  a  process  that 
was  only  politics  "  We  viewed  the  opportunity  to  help  the  Army  transform  and  save 
resources  in  a  time  of  war,  as  well  worth  the  professional  and  personal  risk. 
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jUi  What  did  we  help  the  Army  do? 

*  Supported  IGPBS  and  Modularity 

o  Validated  previous  temporary  stationing  of  Modular  Brigade 
Combat  Teams 

o  Adivoted  Modular  Brigade  Com»bat  Teams  at  three  locations, 

*  Transformed  Institutional  Training 

o  Maneuver  Center 

□  N  el  Fires  Center 

□  Combat  Service  Support  Center 

*  Transformed  Reserve  Component 

q  1 25  Armed  Farces  Reserve  Centers 


BKAL  2tH)5  supported  Army  transformation 
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A.5  LOGISTICS  AND  ACQUISITION 
A.5.1  Cost  Estimating  for  Major  Acquisitions 


INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Cost  Estimation  for  Major  Defence  Systems  Acquisitions 

Faculty 

Name(s) 

Han  de  Nijs,  ACT,  NATO 

Length 

20  -  30  minutes 

Learning 

Objectives 

Understand  the  use  of  hierarchical  clustering  and  decision  trees  to  effectively 
forecast  costs. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Appreciate  the  OA  tools  available  to  forecast  costs. 

Understand  the  hierarchical  clustering  of  costs  of  similar  ships  and  use  of 
decision  trees  to  estimate  costs  for  typical  configurations. 

Content 

Outline 

Case  Study:  The  Netherland’s  Landing  Platform  Dock  (LPD)  ships. 

Data  sample  and  normalization  process  viewed  through  OA  lens. 

Application  of  decision  trees. 

Ex-post  comparison  of  analysis  results. 

Materials 

List  Required  Pre-Readings: 

None. 

Methods 

80%  Lecture  20%  Class  discussion. 

%  Break  Outs. 

Exercise(s) 

None. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

In  your  experience,  where  has  hierarchical  clustering  of  costs  and  use  of 
decision  trees  been  used  to  estimate  costs? 
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Cost  Estimation  for  Major 
Defence  Systems 
Acquisitions 

Hierarchical  Clustering  and  Decision 

Trees 

Han  de  Nijs 
ACT 

t  Oaf-  far  SAS-WS,  3TG-043  7*720 '2  •  J 

Overview 

1.  Situation:  Uncertainty  of  Cost  Estimates  for  a  Major 
Defence  Investment:  Landing  Platform  Dock  (LPD) 

2.  Decision:  Gathering  Cost  Data  from  Other  Nations 

3.  Objective:  Forecasting  Costs 

4.  OA  Contribution:  Hierarchical  Clustering  of  Costs  of 
Similar  Ships  and  use  of  Decision  Trees  to  estimate 
costs  for  typical  configurations 

t  Oraf-  far  SA&-QPS,  5TG-042  7,9/2012  •  2 
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Background 

•  The  Netherlands'  Landing  Platform  Dock  (LPD)  ships: 

a  Rotterdam  LSOC 

*  Commissioned  1997 

□  Joh  a  n  d  e  W  rtt  LSC1 

*  Commissioned  2000 


* 


* 


*JM  Data  Sample 

*  Database  of  descriptive,  technical,  and  cost  information  on  57  ships  in  ISclases 
from  6  nations: 


□  Raleigh  (LPD,  USA) 

o  Thom  as  ton  (LSD,  USA) 

o  Anc  ho  rage  (LSD,  USA) 

o  W h  id  b ey  Isla  nd  (LSD,  USA) 

o  Largs  Bay  (LSD,  UK) 

a  pluscrtfie"  ssngle-slirp'' dsiMs 


-  Mistral  (AAS,  France) 
-Austin  (LPD,  USA) 

-  Pratecteur  (AOR,  Canada) 
-Albion  (LPD,  UK) 

-  Ocean  (LPH,  UK) 


*  Sample  of  the  136  technical  attributes  collected  per  ship: 


□  Vesse  I  type,  d  i  me  nsions  (e .g.,  I  e  ngth  ,  b  earn ),  p  erfo  rmance  (e  g.,  sp  eed,  range ), 
manning,  propulsion  and  paver  gene  ration,  lift  capacity  (e.g.,  we  II  deck,  cargo  space, 
cranes,  elevators),  flight  deck,  armament  and  countermeasures,  data  systems  Irad a r, 
sonar,  etc.), 

a  Both  n  u  m  e  ric  and  nom  in  al  attri  butes 

□  Any  m  issing  data  was  rep  lac  ed  w  rth  d  at  a  e  lem  enf's  average  fo  r  knavn  sh  ips 

□  Principal  component  analysis  reveals  that  the  data  set  can  be  mapped  to  a  15- 
dimensional  space  (accounting  for  9  5%  of  variance  in  original  data) 
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H  Data  Normalization 


*  Cost  data: 

□  Original  data  in  various  currencies  and  base  years 

o  U.S.  data  normalized  usir^  historical  index  for  ship  building  inflation 

□  No  n-U  .S.  d  ata  no  no  alized  using  Eu  rostat  i  ndices 

□  Fictitious Ji not ia n ako mmon  currency'''  |NCC) 

Hiriofnim  of  Cortn  in  NATO  0~6  Ship  Dutibm 


•  JJj* 

OA  Contribution 

•  The  spread  of  the  normalized  cost  was  too  wide 

•  Actual  configurations  of  the  ships  differed 

•  Hypothesis:  costs  depend  on  configuration 

•  Solution:  construct  a  decision  tree 


*  rco_er  Te.v 


7/P/20i3  ii 
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™:hodology:  Decision  Trees 

•  Statistical  Decision  Trees 

>  Combine  decision  trees  and  linear  regression 

>-  Each  node  or  leaf  its  an  estimated  multiple  Inear  regression  equation 
>-  First-ever  application  in  defense  cost  analysts  of  acq  u  ston  prog  rams 


BIS  hodology:  Decision  Trees 


•  Applied  to  Rotterdam  and  Johan  de  Witt  ships 

>  Each  regarded  as  separate  ship  class  by  Royal  Netherlands  Navy 

>  Each  f  o  llo  w  s  a  me  p  ath 


H  of  LCAC  in  Well  Dwk 


LCAC  Landing  Crat  Air  Cusrtfcn 

*  * 
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™thodology:  Decision  Trees 


Applied  to  Rotterdam  and  Johan  de  Witt  ships 


LM3 

LO^COSl ) : 


7,6222 

-0.0167  x  mnfc  in  class 
+  o.ocmi  x  length  (m) 

-  0.0002  x  range  (sailing  lime  in  hrs) 
+  0.0445  x  #  of  LCAC  in  well  deck 
+  0,0659  >:  #  of  torpedo  decoys 


Rotterdam 

1 

162,2m 
500  hrs 
0 

1 


Johan  de  Witt 

1 

175.35m 
S33  hrs 
0 

1 


Cost  estimates: 

Rotterdam  L8QQ: 
Johan  de  Witt  L801: 


197.7M  NCC 
212.3M  NCC 


gj  Results:  Ex-Post 


*  Johan  de  Witt  LPD  estimated  cost  and  actual  cost 
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A.5.2  Replenishment  at  Sea  Scheduling 

INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Replenishment  at  Sea  Planner:  A  Study  in  Applied  Optimization 

Faculty 

Name(s) 

Jeff  Kline,  Naval  Postgraduate  School,  Monterey,  California,  USA 

Length 

60  -  90  minutes 

Learning 

Objectives 

Understand  the  concept  of  optimization. 

Appreciate  the  strengths  and  weakness  of  optimization. 

Understand  through  case  example  the  use  of  optimization  in  logistics 
scheduling. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Appreciate  the  uses  of  optimization  in  logistics  scheduling. 

Content 

Outline 

Case  Study:  Replenishment  at  Sea  Planner. 

Optimization  Defined. 

RASP  discussed. 

Case  Study:  Force  structure  analysis. 

Materials 

List  Required  Pre-Readings: 

None. 

Methods 

100%  Lecture  0%  Class  Exercises. 

%  Break  Outs. 

Exercise(s) 

Seminar  participants  will  be  asked  to  evaluate  charts  and  outputs  from  each 
case  study. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

What  are  your  experiences  with  manual  scheduling? 

How  might  optimization  improve  that  process? 
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§ 

Using  Operations 
Analysis  to  Schedule 
Naval  Logistics  Ships 


A  Case  Study  in  Optimization 


Overview 

SiFuaFiQn:  In  2G 1 1  the  U.$.  Central  Command's  Naval  Commander 
was  faced  with  losing  one  of  four  combat  logistics  force  ships  )T- 
AOJ  to  support  theater  operations. 

Decision:  Would  the  loss  of  o  logistics  ship  require  change  of 
schedules  for  the  operational  forces  and/or  the  other  combat 
logistics  force  ships? 

Qbfective:  Consume  the  Loss  of  a  Logistics  ship  while  minimizing 
impact  on  operations  and  other  logistics  ships  schedules. 

OA  Contribution:  Used  o  scheduling  model  to  determine  that  the 
loss  of  ono  combat  logistics  force  ship  would  not  impact 
operations  ond  to  minimize  impact  on  logistics  ship  requirements 
to  use  higher  speeds  to  meet  fleet  demand. 
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Situation 


*  Naval  Central  Command  (NAVCENT)  supports 
maritime  operation  through  out  their  area  of 
op  e  ra  tlo  n  s .  D  urin  g  this  time,  f  o  ur  lo  g  i  Stic  sh  i  ps  were 
assigned  to  NAVCENT  to  provide  fuel  and 
consumables  at  sea. 


*  NAVCENT  was  Faced  with  a  decrease  of  one  oiler  in 
their  A  OR 


*  Question:  what  is  the  impact  of  only  having  three 
vice  four  logistic  ships? 


■f  Fooler  T^wl 


7/11/15013*3 


Defining  the  areas  of  mterest: 

NORTH  (Arabian  Gulf  and  Gulf  of  Oman) 

SOUTH  (Gulf  of  Aden r  Horn  of  Africa,  Red  Sea) 

Demarcation  point:  Salalah,  Oman 

Blue  dots:  USS  ship  movements  as  provided  by  OPREP-5  FEEDER  data 
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Operations  Analysis: 


Replenishment  at  Sea  Planner 


(RASP) 

*  What  does  RASP  do? 

o  Schedules  at-sea  replenishment  at  the 
operational  level  of  war 

o  Objective:  minimize  Combat  Logistics  Force 
costs  while  maintaining  supply  levels  on  board! 
customer  units 

*  United  States  Military  Sea  Lift  Command  Sponsored 
Development 


■f 


Partnership  between  MSC  HQ  and  N PS 

Present  CLF  employment  options  based  on  quantitative  analysis 
Reduce  workload  on  CTF  staffs 

Enables  tracking  of  fuel,  provisions  and  ordnance  based  on  combatant  operations 
CLF  Costs:  Fuel  consumption,  port  handling ,  commodity  differentials 
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Problem  Statement 


Using  RAS  Planner  with  CTF  53  data  set  reflecting 
28  days  of  NAVCENT  replenishment  operations, 

a  Determine  the  impact  to  combatant  support  given  a 
reduction  in  NAVCENT  CLF  assets  by  one  T-AO 
o  Evaluate  the  increase  in  overall  CLF  speed  necessary 
to  satisfy  theater  fuel  requirements 

a  Summarize  CLF  consumption  [barrels  burned)  by 
scenario 


Customer  Ship  Fuel  Levels 

With  Reduced  T-AO  Presence 


■  RASP  finds  more  replenishmEnt  opportunities  than  manual  planning 

■  Removal!  of  T-AO  193  causes  no  negative  imp  a  cl  in  NOA  customer 
fuel  levels 

o  Only  8  dOy$  of  oiler  o  volubility  lost  in  Northern  Operating  Ar^o 
o  T-AO  203  able  to  cover  regional  requirements 
o  Removal  of  T-AO  139  has  no  impact  on  NOA  customers 

■f  m 


Continued  use  of  .40  -  1.00  range  on  Fuel  levels  axis  to  highlight  differences  {or  lack 
thereof) 
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nm  CLF  Employment 

(Fuel  Deliveries  in  Barrels) 


El  D-L 

Helms  red 

TAKE  1 

TAKE  1 

TAO  IS? 

TAOl?? 

TAP  203 

TAO  204 

rUnrTh 

OPAREA 

SulAotjl 

South 

OPAREA 

Subcecal 

fatal 

i&prrds] 

Four  Oilers 

Data  Set 

L6..SR5 

13.2C3 

21,24  3 

SO.LDS 

77,344 

6].iZ6 

16D, &RZ 

100,734 

261,337 

TAD  193 

Removed 

J4,73E 

£4,677 

13,744 

104r£06 

7£J23 

153, IEI 

101,  B02 

260,934 

TAD  133 

Re-moved 

IA«5 

w.nas 

W.570 

1W,2H 

3WJ7J 

■  Color  c odin£  specific  to  operalinp  area 

*  Botin  DFM  and  JJ’B  tuel  producis  summarized 

*  Delivery  quantities  independent  of  scenario 

*  Stjit]Ktk  ;il]y  iritsijfnifii  rmh  dift^rt'nLt'.s  in  Tiitiil 
^  Reflects  stable  customn  Rquimnmb 
/  No  '‘bonus"'  points  for  unnecessary  delivery 

Customer  demands  can  be  satisfied  with  fewer  assets 


NOTE:  deliveries  for  "Four  Oilers  Data  Set"  map  to  the  events  on  OP  REP- 5  reports  from  the 
CTF  53  data  net  after  reprogramming  that  schedule  into  RASP. 

Customer  demands  can  be  met  with  three  oilers  and  full  use  of  TAKE  capabilities  ...  at 
SLO  WER  transit  speed  requirements 

NEITHER  coalition  customers  NOR  coalition  logistics  support  force  units  are  reflected  in  the 
analysis. 


E2D 

CLF  Bunker  Fuel  Consumed 


TAKE  L 

TAKi  2 

TAO  189 

TAO  198 

TAO  203 

TAO  204 

Wortn 

OPAREA 

Subtotal 

Soudi* 

QPAREA 

liutr-nlnl 

Total 

j&SITils 

Burned^ 

Four 

Oilers 

Z.&OH 

4,642 

2,044 

1,049 

5,562 

5,27? 

11,253 

7,029 

19,182 

TAO  138 

Rcmewd 

3,012 

5,205 

2,116 

3,391 

5,367 

3r596 

8,494 

17,090 

Ti’iO  139 

Removed 

3,012 

Sr205 

1,076 

2,244 

5,222 

8,525 

Sr254 

16,758 
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Conclusion 


*  Reduction  of  a  single  T-AO  in  the  NAVCENT  region 
inflicts  no  adverse  impact  to  combatant  support 

o  Given  the  assumptions  in  RAS  Planner 

*  RAS  Planner  scheduled  more  replenishment 
activities  than  manual  methods  with  fewer  available 
oilers 

o  Additional  RAS  events  improved  overall  theater  fuel  levels 

*  No  speed  increase  observed  to  satisfy  customer 
needs 

o  Reduction  in  "idle  days"  indicates  more  movement  by  CLF 


Analytic  questions: 

Determine  the  impact  to  combatant  support  given  a  reduction  in  NAVCENT  NFAF 
assets  by  one  T-AO 

Evaluate  the  increase  in  overall  NFAF  speed  necessary  to  satisfy  theater  fuel 
requirements 

Summarize  CLF  consumption  (barrels  burned)  by  scenario 
Caveats: 

Observed  behavior  in  data  set  is  not  always  explained  by  replenishment 
requirements 

Reduction  in  CLF  operational  flexibility  not  captured 
Neither  Coalition  forces  NOR  Coalition  CLF  assets  reflected  in  analysis 
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What  is  Optimization? 

•  Field  of  applied  mathematics  that 
seeks  to  find  the  most  cost  effective  or 
best  performing  alternative  under 
given  constraints  by  maximization  of 
desired  factors  or  minimizing  undesired 
ones. 

•  In  RASP’s  case,  to  minimize  CLF  costs 
while  meeting  battle  ship  demands. 


Strengths  and  Weaknesses  of 
Optimization 

*  Seen  as  an  “If”,  “Then"  statement:  "If" 
this  data,  "then”  this  is  the  best  course 
of  action:  need  good  data. 

*  Optimizing  versus  Satisficing:  Ideal 
schedule  versus  reality 

•  Great  asset  for  quick  "What-lf”  analysis 

#  m 
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RASP  Inputs 

*  “Customers11: 

o  Battle  Group  locations  at  certain  days 
o  Demand  for  Fuel  based  on  operations 
o  Minlmums  for  Fuel 

*  “Suppliers'1 

o  Loading  capacities  and  numbers 
o  Locations 
o  Speed 
o  Supply  bases 


A  couple  screen  shots 


RASP  Outputs 

*  Dashboard 

a  Global  view 

*  Animations  for  schedule  review 

o  Sawtooth  by  unit  or  type  fCLF  or  customer) 

*  Reports 

o  Replenishment  Schedule  of  Events 
o  Cost  Summary 
a  Fuel  Consumption  overview 
o  Performance  Metrics 

*  Customer  Ship  Regriiness 

*  CLF  Fuel  Usage 

*  CLF  Maintenance 

*  CLF  Overtime  Labor 


A  couple  screen  shots 
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Customer  Plans  Screen  Shot 


^  C  i! 


Performance  Metrics  Screen  Shot 
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RASP  Take-Aways 

*  Operational  planning  tool 

q  Optimized  replenishment  schedule 
o  Not  applicable  to  every  organization  /  CONORS 
a  Extremely  adaptive  to  planner  requirements 

*  Analysis  tool 

o  Provides  simulation  capability 

*  Enables  quantitatively-based  excursion  assessments 

*  Administrative  tool 

a  Screen  shots  provide  briefing  assistance 
o  Data  capture  atlows  improve  database 
management 

a  Report  generation  capability  to  reduce  staff 
workload 


Methodology  for  the  Analysis 

*  Establish  applicability  of  RAS  Planner  (RASP) 
o  Compare  QPAREa  customer  mean  fuel  levels 

*  Run  RASP  after  removing  T-AO  193 
o  Reduces  8  days  shuttle  oiler  coverage 

*  Run  RASP  after  removing  T-AO  189 

a  Restore  T-AO  1 98 

o  Reduces  28  days  shuttle  oiler  coverage 

*  Evaluate  regional  CLF  availability 


■f  m 
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Reminder:  TAG- 19 8  active  for  only  9  days  during  time  horizon 


Reminder:  TAO-189  active  for  full  28  days 
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NORTH  OPAREA  baseline:  model  tends  to  underforeeast  customer  fuel  levels,  meaning  that 
fuel  state  In  reality  is  BETTER  than  RASP  indicates. 

TAKE  AWAY:  R AS  P  eff e  cti  ve  ly  m  odel  s  c  usto  rn  e  r  fu  el  co  nsu  rn  ption 


RASP  provides  VERY  GOOD  pred  Of  lions  of  customer 

fuel  levels 


SOUTH  OPAREA 
Average  Customer  Fuel  Levels 


RASP  overestimates  Fuel  consumption  by  2.0%, 
exhibits  a  slight  error  (6.4%)  compared  to  Actual 


Data 

&  ^  #  &  #  &  ^  P  J  ,_p  ,tP  P  p  ,-p  .p  p  p  P  P  ■?  P  &  P  P  P  _P 

y  yy  y  y  y  y  y  yyf>y^yyi^yfy^yyJp^^>y^yyfy#^y'^^yt 


^□PREP  &  Values 


-RASP  Estimate 


SOUTH  OPAREA  baseline:  similar  outcome  as  NORTH.  While  NOT  AS  ACCURATE  AS  NORTH 
(smaller  units  cause  greater  variability  -  "skipped"  fueling  event),  still  a  VERY  GOOD 
approximation  to  reality.  Under-estimation  is  better  than  overestimation. 
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CLF  Employment 


(Fuel  Deliveries  in  Barrels) 


asb 

&diimrrd 

TAKC1 

TAKE  2 

TAO  15? 

TAO  155 

TAD  203 

TAO  2(14 

North 

OPAREA 

SoLlotuI 

inuth 

□PAH  EA 

Submeal 

Total 

Itenith} 

Four  Oilers 

DjIj  Set 

16JGS 

13,2  03 

23,2-43 

SG.l&S 

72,344 

SU2& 

IB 0^651 

100,234 

261,397 

TAD  133 

Removed 

J4..73B 

54,677 

13,744 

LD4.5GS 

71.323 

159,111 

101,992 

260,934 

TM>  U3 

Rr-mavrd 

Jl4^I5 

W.520 

fG.cai 

ai.4B2 

15»,^S5 

I0Q..2S2 

2%*A7$ 

1  CoIdt  coding  sped  tic  to  operaling  area 
■  Lkith  DEM  n nd  fuel  piod  ucIe  summarized 

*  Deli  VEty  quanti  ties  independent  o(  scenario 
■S'  S+iliHtic&lljr  irtsi^ni  fiiimt  di:Ftf  rt,nLtjL;  in  Total 
^  Reflects  stable-  customer  requirements 
/  No  "bonus'"  points  for  unnecessary  delivery 

Customer  dem-inds  can  be  satisfied  with  fewer  assets 


NOTE:  deliveries  for  "Four  Oilers  Data  Set',r  imap  to  the  events  on  OPREP-5  reports  from  the 
CTF  53  data  set  after  reprogramming  that  schedule  into  RASP. 

Customer  demands  can  be  met  with  three  oilers  and  full  use  of  TAKE  capabilities  ...  at 
SLO  WER  transit  speed  requirements 

NEITHER  coalition  customers  NOR  coalition  logistics  support  force  units  are  reflected  in  the 
analysis. 
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S  T-AO  Employment 


(Operating  Hours) 


Operating 

|ib  y  Spec  rflji 

TAG  isjy 

TAO  19S 

imwrsi 

TAG  Ml 
(HPUlfSj 

TAD  204 

IHoursh 

Scenario 

syfetrtah 

(Mauril 

Average 
Speed  (ht^|- 

□HV1  Burned 

Four  Oilerb 

Idle 

450 

M4 

244 

413 

1276 

U 

11932 

Data  Set 

Overall  eed 

4 

0 

92 

12 

104 

34.7 

- 

Ti"o  m 

Idle 

J54 

134 

26J 

336 

4.0 

SB74 

RhtkivhJ 

Qvcrspeed 

4 

12 

44 

30 

33.S 

-MS6 

TAO  139- 

Idle 

91 

198 

224 

511 

3.2 

8542 

Rcrn-j'icd 

Gvenpeed 

<1 

4 

36 

40 

39 J 

Employment  results  of  the  three  scenarios  displayed  in  table  format,  featuring  the  number 
of  hours  for  each  unit  either  spent  idle  of  requiring  transits  in  excess  of  20  knots  (a  feature 
that  allows  RASP  to  reach  a  solution  with  crashing  due  to  Planner  manual  inputs.) 

It  is  worth  highlighting  that  28  days  of  activity  amount  to  a  maximum  hours  of  scenario 
time  of  672.  Sor  that  T-AO  189  spent  450  hours  of  that  period  idle  indicates  a  significant 
amount  of  slack  -  or  unaccounted  for  time  with  NO  speed  requirements  -  in  the  schedule. 
A  follow  on  slide  reflects  these  values  in  terms  of  percentage. 

The  average  speed  column  shows  both  the  overall  weighted  mean  speed  [including 
overspeed)  required  to  execute  the  RASP  solution  AND  the  weighted  mean  of  the 
Overspeed  time.  These  values  allow  for  a  comparison  with  the  relative  efficiency  of  the 
schedules.  The  TAG  189  scenario  has  almost  1/3  the  Overspeed  hours  but  requiring  a 
slightly  faster  speed  to  realize  the  solution.  NOTE:  overspeed  conditions  are  only  created 
due  to  Planner-inserted  replenishment  events. 
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Non:  1 1 

L: 

f 

iJlASi 

Operating  Status  Explained 

•  “Idle"  status  reflects  periods  of  relative 
inactivity: 

o  At  sea,  loitering  for  customers 
c  in  port,  typically  cargo  loading 
c  Exhibits  "slack'"  in  the  RAS  schedule 

*  llOverspeed”  indicates  a  speed  requirement  in 
excess  of  ship  capability  to  arrive  at  a  solution 

c  RASP  tolerates  manually  entered  events  causing 
overspeed  conditions  (otherwise  Infeasible) 

c  Overspeed  time  is  highlighted  on  fuel  curve  graphic 
c  Loosened  constraint  enables  planner  to  identify  and 
revisit  potentially  problematic  RAS  assignments 


H!  Operating  at  Extremes 


(Percent  of  lime  Idle  or  Overspeed) 


Operating 
Conditions 
(by  Speed 1 

TAO  tag- 
(%  Timet 

TAO  19S 
(%  Time) 

TAO  203 
Timet 

TAO  203 
(%  Time} 

Scenario 
Subtotals 
[%  Time) 

Four  Oilers 
(Data  Set) 

id(« 

71% 

70% 

46% 

60% 

61% 

OvGrspt-Grf 

2% 

0% 

17% 

2% 

5% 

T&019S 
Ke  moved 

Mle 

47% 

50% 

39% 

37% 

qtygr speed 

1% 

1% 

TAO  ms 

Removed 

idle 

33% 

37% 

33% 

35% 

QGarEpead 

0% 

1% 

5% 

3% 

■  '  ldlcL ;; I ii It. i ^  rcflecl?  pc-nad?  of  rdahvt-  innrtiviTv: 

. M.  t.-j.  l.'  r-.'iiiiL.  liu  .  LE-l-iiiK'i’- 

-  Ill  pOTL  IVpiCJlV  Mrtfj  khfclii'ti 

-  'sltth"  InOrr  KA5  KteMt 

*  “HJlflflSphrtT  il'KiiL'.’ili-B-  .'i  ApMhJ  in  or  iJTlp  il1 ,i|Vll  '.lil y  lit  .11  tl  W  A I  .1  h  Ail  Hin 

K-1SI1  M  Ip  rim  m  ininlh  rnlrrvvd  ■  ■’.■unli  mnHitkwi^  ij^hpiww  imr  j-nr-ii-i 

ttaerr-pt  nJ  linv  it  im  KiH  ffluv  u r.  1 1 i 
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The  overlaid  graph  is  to  show  more  detail  to  the  overall  speed  requirements  of  each 
scenario  without  being  overwhelmed  by  the  IDLE  hours  data,  which  is  captured  in  the  side 
graph. 

The  fuel  curves  indicate  a  very  shallow  efficiency  valley  for  the  KAISER -class  fleet  oiler  from 
about  7  knots  to  13  knots,  reflected  above  in  RASP  utilization  of  those  lower  speeds  to 
achieve  It's  schedule  requirements. 


jjjj  The  Flaw  of  Averages 


*  "Bubble"  area  proportion^]  to  hours  Fit  fipwd 

*  Changes  in  average  ^peed  not  nece^na  rLly  indicati  ve  of  best  cubt 

-■  Moving  at  6  kn&ls  is  more  expensive  than  1 0  knots 

-  Increasing  T>  kts  to  15  kts  (-+15^0  increases  cost  from  SHfl  ptrr  roik  fa  {-*25%) 
1  Thie I  consumed  is  $  barter  far  cost  snely&is 
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T-AKE  Performance 


T-AKE  1 

RAS 

IDFM] 

RAS 

(IPS) 

Load 
(Of  M) 

Load 

(JPS) 

UN  REPS 

INGE PS 

Four  Oilers 

{13,702) 

(2,562) 

- 

3 

I 

TAO  193 

Removed 

{11,921) 

(2,314) 

- 

- 

4 

1 

TAO  Itt 
Removed 

{12,007) 

(2,317) 

- 

4 

1 

Recall: 

T-AKE  1  operated  In  the  SO  A,  supporting  mostly  small  combatants 

T-AKE  2  operated  in  the  NO  A  in  direct  support  of  the  CVN  as  well  as  the  LHD  in  transit 

T-AKE  2  breaks  out  "Stores  Only"  UN  REPS  to  showcase  the  underutilization  of  the  refueling 
capability  in  the  fixed  event  scenario;  however,  two  additional  JP5  hits  given  to  CVN 
without  stores  in  the  other  scenarios  may  not  be  acceptable  practice. 
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T-AKE  Performance 


T-AKE  2 

RAS 

(DFM) 

HAS 

(JPS) 

Load 

(DFM) 

Load 

(IPS} 

UNREPS 

UNREPS 

{slates.) 

INREPS 

Pour  Oilers 

(27,963) 

(6,236} 

15,112 

736 

12 

7 

4 

TAO  198 

Removed 

(39,336) 

(14,340} 

26,375 

10,390 

11 

2 

4 

TAO  169 
Rermjvnd 

(40,153) 

(14,840} 

27,192 

10,390 

ii 

1 

4 

Recall: 

T-AKE  1  operated  in  the  SO  A,  supporting  mostly  small  combatants 
*  T-AKE  2  operated  in  the  NO  A  in  direct  support  of  the  CVN  as  well  as  the  LHD  in  transit 

T-AKE  2  breaks  out  "Stores  Only"  UN  REPS  to  showcase  the  underutilization  of  the  refueling 
capability  in  the  fixed  event  scenario;  however,  two  additional  JP5  hits  given  to  CVN 
without  stores  in  the  other  scenarios  may  not  be  acceptable  practice. 


Hull  Presence  (CLP) 


CLF  Activity 

ffl 

CLF  Presence 

RciSc»  SjifljUfc-i  -BuroiOffin 
■  riiknrh  ■WpT-aOift 


2  « 


1“ 

Jli 

V 


kli 


HcdScj  Gul^Adcti  ■Gtilftifaman  Arataiantiulf 
FulliM-narta  s  Less- AG  1®  lessTAG  LSr 


RASP  schedules  additional  events  to  satisfy  of  customer 
requirements 

a  Despite  fewer  days  of  CLF  avclabilily,  more  RA5  even  Is  were  conducted 

In  Data  Set.  T-AO  203  surges  from  Gulf  of  Oman  to  Gulf  of  Aden  fm 
,9-13  April 
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Spike  in  Arabian  Gulf  support  due  toT-AG  203  Rationing  in  NOA  for  duration  of  period,  viice 
four  day  sortie  to  SOA  in  Fixed  scenario. 
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A.5.3  Out-Planning 


INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Optimization  of  OUT-Planning  in  the  Context  of  Contingent  Rotations 

Faculty 

Name(s) 

Bundeswehr  Planning  Office,  Ottobrunn,  DEU 

Length 

20  -  30  minutes 

Learning 

Objectives 

Understand  that  OR-methods  can  improve  complex  planning. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Recognize  similar  applications  where  an  analysis  by  OA  staff  could  support 
planning. 

Content 

Outline 

Case  Study:  Optimization  of  OUT-planning. 

Materials 

None. 

Methods 

100%  Lecture  0%  Class  Exercises. 

%  Break  Outs. 

Exercise(s) 

None. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

Did  you  have  a  similar  case  in  the  past? 

Can  you  think  of  a  current  case  in  your  area  of  responsibility? 
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OUT-Planning 
of  contingent  rotations 


A  Case  Study  in  Optimization 


Name  Lecturer 

Bundeswehr  Planning  Office 


*1 


Overview 


Situation:  Military  airlift  fly  troops  from  TERMEZ  to  COLOGNE 
Planners  have  l he  option  to  plan  for  one  stop-over,  provided  (fiat  a 
minimum  of  50  soldiers  want  to  disembark  al  this  airport.  Soldiers  are 
flexible  within  +-/-1  day  of  planned  outdate. 

Decision:  maximize  the  number  of  soldiers  whose  airport  requests  can 
be  realized 

Objective:  optimize  flight  schedule 

OA  Contribution:  To  analyze  and  optimize  a  system 

Result  or  outcome:  Increase  the  number  of  realized  airport  requests 
exceeding  100% 


•  2 
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Agenda 


>  Context  (OR  support  of  ISAF) 

>  Problem  statement  “OUT-Planning" 

>  Ways  of  problem  solving 

>  Results 

>  Conclusion 


*■3 


OR-Support  to  ISAF 


I 


Operations 

Command 


Subject- 
s pacific  lines  of 

operations 


Organisation 

PfararPiTcr  Offlrfce 


Military 

LbMl&r 

TVim 

Add  P-irawnil 

* 

UnF.v. 

expiri¬ 
n' i'-n  ri 

External  Gompanlas 

Operational 

linei  of 

operations 


The  Bundeswehr  OR  support  to  ISAF  operations  has  been  provided  since  2007. 

As  you  can  see  on  the  slide,  OR  support  to  operations  is  characterized  by  close 
collaboration  and  co-ordination  between  the  German  Joint  Forces  Command,  the 
OR  cell  in  Afghanistan,  and  the  Reach  back  organization  in  Germany. 

The  OR  cell  is  the  focal  point  in  theater.  It  gathers  inputs  for  future  analysis  and  also 
conducts  “first-line"  analysis.  If  needed,  the  Reachback  organization  supports 
further  analysis  by  providing  research  support  or  serving  as  a  coordination  element 
with  other  national  military  or  scientific  organizations  in  Germany.  The  Joint  Forces 
Command  serves  as  the  operational  headquarters  for  all  Bundeswehr  operations 
abroad. 
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^  Problem  Statement 
OUT-Planning 

o  During  contingent  rotation  several  thousand  soldiers  are  to 
be  flown  outvie  Mazer-e  Sharif  to  Germany 

□  The  airplane  flees  to  Cologne  on  a  regular  basis;  on  demand 
(min.  requirement:  50  people)  a  stop-over  can  be  planned 
(e.g.  in  Berlin,  Hannover,  Stuttgart  Nuernberg) 

o  Every  soldier  can  select  the  airport  that  is  closest  to  his 
home  (-base) 

o  Assumption:  the  out-date  can  be  changed  by  +/-  one  day 

o  Aim-  maximize  the  number  of  desired  airport  requests 


*? 


Results  of  manual  planning 


□  The  Excel  template  for  the  flight  planner  is  very  complex 
and  rarely  suited  for  manual  optimization 

o  The  figure  shows  the  cumulated  data  of  a  past  OUT- 
planning  result 

o  On  several  Consecutive  days  the  chance  for  Hannover  was 
closely  missed 


The  flight  planner  uses  an  Excel-Sheet  for  manual  planning.  The  problem  is  too 
complex  to  quickly  find  an  optimal  solution  just  by  working  with  passenger  lists  on  a 
manual  basis.  The  depicted  figure  shows  a  planning  overview  where  the  bound  of 
50  people  for  a  stop  in  Hannover  was  closely  missed. 
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§  Solution  Approaches 

c  Approach  1: 

VBA=Prog#affi  lhat  Improves  the  assignment  ot  soldiers  ta  speoltie  &u1  dates 
b assd  on  si  simple  heuristic* 

■r  Seamless  Integration  Into  Excel  workbook 
o  but; 

•r:  Herd  to  Judge  the  quality  of  the  computed  solution 

o  Optimality  is  extremely  unlikely 

<r:  Ef  rot-prone,  high  maintenance  eliort  expected 

■0  Approach  2; 

algebraic,  modeling  of  the  problem  and  application  of  professional  compuler 
programs  for  calc  u  lesion 
o  Botler  solutions  expected 

ts  Re-use  of  existing  methods  end  standard  algorithms 

o  but; 

c.  Requires  export  and  import  of  the  data  as  well  as  the  computation  of 
the  soldi  ion  using  special  equipment 
o  Large  modal,  many  variables 

fe  *  10 


Two  approaches  where  realized,  but  only  the  second  one  really  found  the  optimal 
solution.  The  first  is  a  greedy  heuristic  realized  via  Virtual  Basic  for  Applications 
(VBA)  in  Excel.  This  was  easy  to  integrate  to  the  Excel-Sheet  of  the  flight  planner 
but  cause  of  the  heuristic  approach  it  is  difficult  to  judge  the  quality  of  the  solution. 
The  error-prone  is  very  high. 

The  second  approach  is  to  model  the  problem  algebraic.  The  advantage  is  that  you 
can  use  existing  and  standard  algorithm  to  solve  the  problem.  The  quality  of  the 
solution  should  be  better,  but  you  have  to  export  and  import  data  from  the  Excel- 
Sheet  to  compute  the  solution  with  a  separate  tool.  Due  to  the  complexity  of  the 
problem  it  is  not  linear  and  the  model  grows  very  fast  because  you  have  many 
variables  and  restrictions.  The  computation  time  could  enlarge. 
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'aS  Model  (1) 


d  Set  of  soldiers 
o  Set  of  OUT-days 
u  Set  of  stop-over  airports 

a  Capacity  of  airp lane 
o  Airports  requested  by  soldier 
o  Pcss  i  bie  OUT^ley s  cf  sold  ier 

o  Flight  plan 
o  OUT  plain 

a  Number  of  soldiers  satisfied 


sets 


conslanls 


variables 


iarget  function 
tiappkneos 


*n 


'JUS  Modell  (2) 


o  Every  soldier  is  flown  out  exactly  once, 
o  The  assigned  date  has  to  be  possible. 


o  The  capacity  of  the  aircraft  must  be  taken  into 
account 


o  Maximum  of  one  stop  per  flight. 

o  If  there  is  a  stop,  at  least  50  soldiers  have  to  debark, 

»  *12 
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Implementation  Details 

o  Assumptions  of  the  model  have  been  validated  by  OR  analysts 
and  flight  planners  in-theater 

o  Using  a  common  Algebraic  modeling  tool  for  the  computation 
of  solutions 

o  First  experiments  by  Rcachbaclk  team  with  two  open  source 
solvers,  but  these  did  not  manage  to  come  up  with  probably 
optimal  solutions  in  a  reasonable  time  due  to  the  large  size  of 
the  input  models 

o  Still  for  tlie  example  data  of  past  OUT  plannings  the  computed 
solutions  were  +60  better  than  the  heuristic  approach  and 
magnitudes  batter  than  the  result  of  the  planning  by  hand? 


i  *13 


§  Quality  of  Solutions 
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Here  you  can  see  a  comparison  of  the  planning  by  hand  and  the  planning  with 
approach  2.  As  you  can  see  an  improvement  is  possible.  The  number  of  soldiers 
which  airport  request  could  be  realized  doubled. 
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Challenges 


o  OUT  days  must  not  change  on  a  short  notice 

o  Number  of  changes  in  OUT  days  should  be  as  low  as 
possible  to  remain  feasible 

o  Solution  needs  to  be  robust  against  flight 

cancellations  or  short-notice  changes  of  OUT  days 
for  various  reasons 

o  Cost  (for  stop-over)  should  be  considered!  at  least  as 
a  secondary  criterion 

>  *  II- 


Lessons  Learned 


o  Surrounding  conditions  change  continually 

^  assumptions  need  to  be  checked  not  only  at  the 
beginning  but  again  and  again 

o  Constantly  keep  in  touch  with  the  user 

o  Convince  key  personnel  in  order  to  get  them 
involved  as  promoters 


•  «i? 
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Conclusion 

o  Project  has  the  potential  to  improve  the  satisfaction 
of  our  troops  and  to  promote  the  application  of  OR 
methods 

o  Flight  planning  is  a  classical  optimization  problem, 
this  problem  touches  on  is  a  new  aspect 

o  Different  approaches  and  models  had  to  be  explored 
in  order  to  achieve  a  tangible  result 

o  Professional  tools  are  required  for  productive  OR 
work 


Impressum 


•  Created  by:  Bundeswehr  Planning  Office 
■  References:  TBD 

•  Materials:  TBD 

•  Lecturer:  LTC  Burgert 


•  I'D 
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A.5.4  Stockpile  Planning  Guidance 

INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Methodology  to  Derive  Stockpile  Guidance  for  NATO  MC  55/4 

Faculty 

Name(s) 

Han  de  Nijs,  NATO  HQ  SACT,  Norfolk  VA,  USA 

Length 

45  -  60  minutes  Lecture;  Additional  30  minutes  Exercises  and  Discussion 

Learning 

Objectives 

Appreciate  the  value  of  simple  models,  assumptions  and  conclusions  that 
can  be  immediately  grasped  by  decision-makers. 

Appreciate  the  need  for  quick  analysis  that  is  relevant  for  a  problem  that  is 
imminent  at  NATO  Committee  levels. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Understand  the  impact  analysis  can  have  on  investment  in  stocks  and 
material  by  the  NATO  Nations. 

Understand  the  power  of  analytical  models  that  are  simple  but  relevant. 

Understand  the  need  for  analysts  to  be  familiar  with  subject  matter. 

Content 

Outline 

A  Case  Study  of  application  of  simple  model  for  the  determination  of  NATO 
stock  levels  to  be  maintained  by  forces  to  be  deployed  in  NATO  operations. 

Materials 

List  Required  Pre-Readings: 

NATO  Logistics  Handbook,  November  2012: 
http://www.nato.int/docu/logi-en/logistics_hndbk_2012-en.pdf 

Methods 

60%  Lecture;  30%  Breakouts  for  Exercises;  10%  Class  Exercises; 

Discussions 

Exercise(s) 

Estimate  cost  savings  in  stock  levels  for  Class  1  and/or  Class  II  for  the  new 
policy  for  armed  forces  of  your  Nation. 

Estimate  lead  times  for  the  production  (either  in  home  country  or  in  host 

Nation)  of  some  classes  of  stocks  (Class  1,  II,  III  and/or  IV). 

Determine  situations  where  you  think  the  re-supply  time  T2  would  make  a 
difference  and  cannot  be  neglected. 

Are  all  assumptions  always  right? 

Break  Outs 

X  Yes  No. 

#  5 _ participants  in  each  break  out  group. 
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Discussion 

Questions 


Discuss  questions  arising  from  exercises? 

Other  questions: 

Is  this  model  too  simple?  What  can  be  improved? 

Although  Nations  welcomed  the  outcomes  on  one  hand,  they  were  not  happy 
with  on  the  results  on  the  other  hand  when  they  applied  it  to  munitions.  What 
is  the  reason  for  that? 

If  this  would  have  been  a  result  of  simulations  would  it  have  had  the  same  or 
a  different  impact  on  decision-making? 


Methodology  to  Derive 
Stockpile  Guidance  for 
NATO  MC  55/4 


Decision  Support 
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S  Overview 


1 .  Situation:  Stockpile  Requirements  for  NATO  nations 
expressed  as  a  simple  30  Days  of  Supply  (DOS)  for  all 
classes  of  supply  is  not  anymore  economic  feasible 

2.  Decision:  Agree  in  NATO  forum  on  new  levels 

3.  Objective:  Reasonable  and  Justifiable  Stock  Levels 

4.  OA  Contribution:  Provide  rationale  for  stock  levels 
required  for  each  class  of  supply  that  includes 
transparent  logic,  easy  compliance  measures  and 
reasonable  burden 


S  Background  (1) 

*  MC-31 7  -  NATO  Force  Structure 
Review1  resulted  in  the 
adoption  of  principle  of 
Graduated  Readiness  to 
provide 

□  flexibility  for  conducting  full  range  of  missions  and 
o  sustainability  to  ensure  collective  defence 

*  NATO  Force  Structure  distinguished 

o  Higher  Readiness  Forces  (HRF| 
o  Forces  of  Lower  Readiness  (FLR) 
o  Long  Term  Build-up  Forces  (LTBF) 

*  Revision  of  Logistic  Readiness  and  Sustainability  MC- 
55/3  was  necessary 

I)  MC  317  of  '  6  Dec  '9f '  was  i  .peri-seed  by  MC  317 f  of  S  July  2002  3 


ANNEX -206 


STO-TR-SAS-098 


NATO 

OTAN 


ANNEX  TO  SAS-098 


H  Background  (2) 

•  MC  55/3:  Requirement  that  nations  have  stockpiles  for  all 
classes  of  supply  for  30  days  for  all  their  forces  declared  to 
NATO. 

•  MC  55/4  Starting  point:  To  optimize  flexibility  of  nations  in 
how  they  meet  logistics  readiness  and  sustainability 
requirements  by  the  end  of  the  specified  readiness 
preparation  time 

•  Change  in  perception:  from  sustainment  to  sustainability,  i.e. 
the  ability  to  sustain  ready  forces  in  time 


NATO  Classes  of  Supply 

•  Class  I  -  Items  of  Subsistence,  e.g.  food  and  forage 

•  Class  II  -Supplies  for  which  allowances  are  established  by 
Tables  of  Organization  and  Equipment  (TOE),  e.g.  clothing, 
weapons,  tools,  spare  parts,  vehicles 

•  Class  III  -  Petroleum,  Oil  and  Lubricants  (POL)  for  all  purposes 
(except  for  operating  aircraft  or  for  use  in  weapons) 

o  Class  Ilia  —Aviation  Fuel  and  Lubricants 

•  Class  IV-Supplies  for  which  initial  issue  allowance  are  not 
prescribed  by  approved  issue  tables 

(e.g.  fortification  and  construction, 
as  wel  I  as  a  d  d  itio  n  a  I  q  u  a  ntities  for 
items  authorized  for  initial  use) 

•  Class  V- Ammunition, 
explosives  and  chemical 
agents 
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Considerations  that.. 

•  Logistics  plans  must  ensure  that  sufficient  quantity  and 
quality  of  logistics  resources  are  available  at  the  same 
readiness  and  deployability  levels  to  support  forces  until 
such  time  as  a  stable,  robust  re-supply  system  has  been 
established 

•  Stockpiling  has  to  take  into  account  the  demand  for  supplies 
that  arise  from  NATO's  ambition  for  operations  and  the 
requirements  for  collective  defence 

•  Reflect  the  trend  towards  integrated  military  logistics  chain 
management  and  the  increasing  importance  of  partnering 
with  industry 

o  Industrialand  commercial  market  considerations 
o  Str  ategi  ctransportationavailability 
o  Access  to  strategic  materials 


.  lead  to  the  bottom  line 


•  Need  to  have  sufficient  stocks  for  High  Readiness  Force  (HRF) 
units  that  are  ready  to  deploy  based  on  the  perceived  need 
for  supplies  in  the  operation  until  a  stable  and  robust  re¬ 
supply  system  is  in-place 

Balanced  against 

•  Requirement  for  stockpiling  to  be  flexible  where  Forces  of 
Lower  Readiness  (FLR)  and  Long  Term  Buildup  Forces  (LTBF) 
are  needed  -  specifically  when  deploying  in  roulement  of  the 
HRF  of  same  nation 
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Determine  Stock  Levels 

*  How  to  determine  agreed  level  of  stocks? 

o  NATO  uses  an  agreed  model  to  compute  Munitions 
Stockpiles 

o  There  are  no  agreed  models  for  other  classes  of  supply 

o  Supplies  to  be  based  on  typical  demand  that  arises  from 
the  NATO  assumptions  for  typical  operations  in  which 
NATO  is  willing  to  engage 

*  Agreement  on  what  is  consumed :  Standard  Day  of 

Supply  (SDOS) 

o  The  total  amount  of  supplies  required  for  an  average  day 
based  on  Standing  Group  NATO  rates  and/or  national 
rates  as  appropriate 

*  Howto  agree  on  what  is  required  for  the  unit  ready 

to  deploy? 


Ek  S  O A  C  ontribution 

Devise  method  with  simple  rules 

*  Based  on  steady  state  in  unit  deployment 

•  Which  will  produce  a  demand  that  must  be 
equaled  by  production: 

o  P=C’X 

o  P=Daily  Production  /  Output  Level  of  Home  Base  Industry 
o  C=Daily  Consumption  rate 
o  X=Number  of  Consumers 

Consider  Battle  Damage  Munitions  separately 
Call  it  the  Equilibrium  Methodology 
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Conclusions 

•  Analysis  determined  that  the  most  important 
parameter  is  the  time  that  industrial  and 
commercial  production  catches  up  with  demand 

•  Demand  is  generated  only  by  consumers  that  are 
deployed 

•  Stock  levels  that  come  from  production  that  is 
sufficiently  early  started  before  a  deployment  need 
not  to  be  maintained  in  peacetime 


Impact 

•  Some  classes  of  supply  are  immediately  available 
from  the  civilian  market  and  industrial  base  and 
therefore  there  is  no  need  to  stock  these  items 

•  While  some  other  classes  of  supply  may  have  a 
lead-time  for  production  to  be  set  up  or  catch  up 
with  the  demand,  and  therefore  stock  levels  are 
necessary,  it  will  only  be  required  to  have  those 
stock  levels  for  High  Readiness  units 

•  However,  munitions  typically  have  a  very  long  lead- 
time  for  production,  leading  to  a  requirement  to 
have  stocks  available  for  much  more  than  one  re¬ 
supply  cycle  and  beyond  the  original  MC  55/3  30- 
days  SDOS 
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What  was  the  Decision? 

•  Only  for  units  in  High  Readiness 

•  30  SDOS  is  the  general  guidance 

•  More  specifically: 

a  Class  I:  No  military  requirement  to  stockpile  fresh  rations;  for 
p  of  able  waterit  isleftto  nations  to  assure  availability  of  30  SDOS 

a  Class  II:  An  30SD0S  is  not  generally  applicable,  except  for  a  few 
items  (NBC  Clothing) 

a  Class  III:  30  SDOS  ready  availability  of  bulk  products 

o  Class  IV:  Mainly  time-independent  and  mission  driven;  30SDOS 
not  applicable 

q  Class  V:  Battle  Damage  Munitions  follow  level  of  effort 
methodology  and  non-BDM  re  quire  merit  is  30  SDOS 


■2 


B M 

Final  Result 

*  The  new  MC  55/4  led  to  a  drastic  reduction  in 
volume  and  cost  to  nations  of  material  to  be 
routinely  stockpiled  to  meet  NATO  readiness  and 
sustainability  needs 
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More  Details  Follow 


U 


R  S  O A  C  ontribution 

Devise  method  with  simple  rules 

*  Based  on  steady  state  in  unit  deployment 

•  Which  will  produce  a  demand  that  must  be 
equaled  by  production: 

o  P=C’X 

o  P=Daily  Production  /  Output  Level  of  Home  Base  Industry 
o  C=Daily  Consumption  rate 
o  X=Number  of  Consumers 

Consider  Battle  Damage  Munitions  separately 
Call  it  the  Equilibrium  Methodology 
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3  Assumptions 

•  Consider  situation  with  longest  deployment 
distances 

•  Assume  national  strategic  lift  is  finite 

•  Assume  units  will  bring  combat  load  with  them,  i.e. 
supplies  that  will  be  consumed  in  first  e.g.  7  days 

•  Assume  that  industrial  and  market  mechanisms  are 
leveraged  in  home  base 

•  Assume  that  a  re-supply  cycle  can  be  established 

•  Take  SDOS  as  the  daily  consumption  rate 

•  Assume  lead-times  for  industry  to  start  production  of 
the  supply  item  thus  necessitating  armed  forces  to 
maintain  stock  levels  for  an  initial  period 


F  ir  st  C  onclus  ions 

•  The  amount  of  supplies  combat  ready  forces  need 
to  have  on  hand  depend  on 

□  Nun  her  of  d  ays  u  nit  nee  ds  to  settle  i n  ’  before  reg  u  ar  su  p  pi y  lines  are  set 
u  p :  for  that  rules  suo  h  as  7  d  ays  of  oon  b  at  ready  su  p  plies  are  oreated 

□  Number  of  days  that  the  transportation  of  sup  plies  lag  beh'ndthe 
deployment  of  the  force;  the  debarkation  of  military  units  at  the 
SPOD/APOD  a  ndthe'r  on  ward  movement  will  have  a  higher  pricritythan 
the  tra nspcrfatfon  of  supplies  to  the  forward  supply  points 

•  The  re-supply  cycle  packages  supplies  in  such  a 
way  that  the  force  is  sustained  for  the  period  until 
the  next  package  arrives: 

a  Packages  will  be  in  bulk  and  boost  the  su  p  ply  levels  whio  h  will  then 
gradually  d'min  sh,  i.e.  Stock  levels  follow  a  typical  saw-tooth  pattern 

•  Only  a  small  portion  of  national  units  are  in  high 
readiness  and  industrial  production  can  easily 
anticipate  or  catch  up  for  units  in  FLR  or  LTBF 
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§  Simple  Model 

•  P  is  daily  production  level  of  class  Z  of  supply 

•  C  is  daily  consumption  rate 

•  X  is  number  of  consumers 

•  P=C*X 

•  T0=  time  in  days  for  which  the  unit  needs  to  take 
supplies  to  theater  before  re-supply  arrives 

•  T !  =  time  in  days  for  production  line  to  be  set  up  to 
reach  output  level  of  P 

•  T2  =  the  number  of  days  of  supply  that  is  in  a 
package  ot  re-supply 

•  N  =  number  of  re-supply  cycles  before  production 
catches  up  with  consumption 


Number  of  Cases 

•  T i  <  T0:  The  production  level  in  civilian  market  is 
adequate  to  provide  consumption  levels  immediately 

□  Sh'pnentof  unit  and  re-sup py  package  are  happen'ng  atthe  same  time 

a  U  nit  ca  n  use  con  b at  re  a  dy  su  p  py  a  nd  then  get  re-su  p  py i n  mediately 

o  Exa  m  pies:  too d  a  n  d  water  (some  excepfons  with  fresh  food } 

•  T0  <  T,  <  T0+T2:  The  production  level  is  providing 
enough  for  consumption  after  the  combat  ready 
supplies  are  exhausted  and  before  the  first  re-supply 
cycle  time  has  passed 

o  Ad  ditibnal  stock  level  to  o  vercone  first  re-su  p  p;y  cycle  ti me 

□  E.g.  Td=10  days  an  dT^l  4  days;  if  production  reaches  level  P  after  lOdays 
but  before  24  days,  then  we  need  stock  for  lOH  4=24  days. 

•  T0+N*T2<  Ti  <  To+(N+1}*T2:  The  production  level 
reaches  consumption  level  equilibrium  only  after  N 
cycles  of  re-supply 
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S  Conclusions 

*  From  the  formula 

T0+NrT2<T,<V(N-Hl}rT2 

we  derive  that  re-supply  cycle  time  T2 is  of  little 
importance  to  determine  time  T1(  the  time  when 
production  levels  reaches  consumption  level 
equilibrium 

*  The  most  important  parameter  is  T |  the  time  that 
production  will  catch  up  with  demand,  demand 
that  is  generated  only  by  consumers  that  are 
deployed 

*  And  that  stock  levels  for  future  consumers  can 
come  from  production  that  is  sufficiently  early 
started,  and  therefore  these  stocks  need  not  to  be 
maintained 


Impact 

•  Some  classes  of  supply  are  immediately  available 
from  the  civilian  market  and  industrial  base  and 
therefore  there  is  no  need  to  stock  these  items 

•  While  some  other  classes  of  supply  may  have  a 
lead-time  for  production  to  be  set  up  or  catch  up 
with  the  demand,  and  therefore  stock  levels  are 
necessary,  it  will  only  be  required  to  have  those 
stock  levels  for  High  Readiness  units 

•  However,  munitions  typically  have  a  very  long  lead- 
time  for  production,  leading  to  a  requirement  to 
have  stocks  available  for  much  more  than  one  re¬ 
supply  cycle  and  beyond  the  original  MC  55/3  30- 
days  SDOS 
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A.5.5  Intra-Theatre  Airlift 

(See  Application  Area  2  -  Section  A.2.3) 

A.6  SCHEDULING 

A.6.1  Replenishment  at  Sea  Scheduling 

(See  Application  Area  5  -  Section  A. 5. 2) 

A.6.2  Out-Planning 

(See  Application  Area  5  -  Section  A.5.3) 
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A.7  OTHER 

A.7.1  Quantitative  Risk  Assessment:  Operation  Eagle  Claw 


INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Quantitative  Risk  Assessment:  It’s  All  About  the  Variability 

Faculty 

Name(s) 

Jeff  Kline,  Naval  Postgraduate  School,  Monterey,  California,  USA 

Length 

20-30  minutes 

Learning 

Objectives 

Understand  the  relationship  between  quantitative  risk  assessment,  variability, 
and  uncertainty. 

Appreciate  the  dangers  of  average-based  decision-making. 

Appreciate  the  strengths  and  weakness  in  simulation-based  risk  assessment. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Appreciate  the  sources  of  variability  and  relationship  to  risk. 

Understand  the  shortcomings  of  average-based  decision-making. 

Read  an  “S”  curve  to  provide  risk  given  a  level  of  resource. 

Content 

Outline 

Case  Study:  Operation  Eagle  Claw. 

Average-based  decision-making  examples. 

Case  Study:  Gas  Turbine  rework  facility  budgeting. 

Surprise  and  Black  Swans. 

Recommend  Readings. 

Materials 

List  Required  Pre-Readings: 

Savage,  S.,  “The  Flaw  of  Averages:  Why  we  Underestimate  Risk  in  the  Face 
of  Uncertainty”,  John  Wiley  &  Sons  Inc.,  Hoboken,  New  Jersey,  ISBN  978-0- 
471-38197-6,2009. 

Methods 

80%  Lecture  20%  Class  Discussion. 

%  Break  Outs. 

Exercise(s) 

Seminar  participants  will  be  asked  to  evaluate  charts  and  outputs  from  each 
case  study. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

In  your  experience,  where  is  average-based  decision-making  used  most  in 
your  organization? 

How  can  variability  be  taken  into  account  while  planning  operations? 
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g.psa 

Applied  Probability 
and  Simulation  to 
Understand  Risk  in 


Operation  Eagle  Claw 

A  Case  study  in  the  Flaw  of 
Average-Based  Decision  Making 


•  Graf  I  lor  RTG-043 


7/15/2013  ■! 


Slide  one:  Introduce  subject  and  instructor 

This  case  study  deals  with  an  application  of  operational  analysis,  risk  assessment,  that  was 
not  done  but  according  to  a  United  States  Admiral  should  have  been  done.  We  look  back 
to  late  1979  when  Iranian  demonstrators  took  over  the  American  Embassy. 
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Overview 


Situation:  Iran.  November  1979,  the  American  Embassy 
was  over  run  by  Iranian  revolutionaries  who  took  52 
Americans  hostage,  In  April  1 930  U.S.  forces  attempted 
a  recuse  codenamed  OPERATION  EAGLE,  It  ended  in 
failure  at  Desert  One  due  to  helicopter  failure, 

Decision:  How  many  helicopters  to  take  on  the  mission 
knowing  six  were  required  to  lift  the  rescue  team  and 
the  hostages? 

Objective:  Take  a  sufficient  number  of  helicopters  to 
keep  the  risk  of  mission  failure  due  to  helicopter  failure 
under  given  limit  (e.g.  5%). 

QA  Contribution:  None  in  the  planning  and  execution 
of  the  operation.  QA  was  engaged  in  the  post  mission 
analysis  to  investigate  on  risk  assessment. 

■  7*11*2013 


Here  is  the  overview  and  situation.  After  five  months  of  captivity.  President  Carter  ordered 
a  rescue  attempt  of  the  Americans  being  held  by  Iranian  forces  Inside  the  American 
Embassy.  That  attempt  failed  due  to  a  catastrophic  accident  that  occurred  at  a  desert 
refueling  site  called  Desert  One.  The  post  investigation  uncovered  several  planning 
challenges  but  the  one  we  will  focus  on  is  the  decision  on  how  many  helicopters  to  use  In 
the  mission.  To  lift  all  the  U.S.  forces  and  return  with  all  the  hostages  a  total  of  six 
helicopters  were  required  to  make  the  round  trip.  The  issue  for  the  original  planning  team 
was  how  many  helicopters  to  send  to  account  for  helicopter  failures.  As  we  will  see,  the 
planning  team  just  used  a  form  of  average-based  decision  making  and  did  not  apply  some 
simple  risk  assessment  tools  to  give  them  an  assessment  of  possible  overall  mission  failure 
due  to  helicopter  reliability. 


STO-TR-SAS-098 


ANNEX -219 


ANNEX  TO  SAS-098 


™  Situation:  Risk  Assessment  in 
Operation  EAGLE  CLAW 

*  Issue  1 1  of  the  "Holloway  Report"  (Operation 
Eagle  Claw's  Planning  Evaluation) 

o  RH-53D  SEA  STALLION  Force  Size  or  Risk  versus 
Resources.  How  many  is  enough?  6?  8?  Or  1 2? 

*  Calculation  based  on  Expected  Value 

o  Historically  reliability  of  each  RH-53D  SEA  STALLION  is 

75% 

o  Based  on  expected  value  8  helicopter  are  needed  (8 
*  0.75  =  6} 

Problem: 

Expected  value  decision  making 


Slide  3: 

After  the  Iranian  Rescue  mission  failure,  a  follow  on  investigation  lead  by  Admiral  Holloway 
uncovered  several  challenges  to  the  planning  and  execution  of  this  operation.  His  report 
lead  to  the  establishment  of  the  U.S.  Special  Operations  Command. 

Our  interest  is  Issue  number  11  from  the  report:  The  way  the  number  of  RH-53Ds  readied 
for  the  mission  was  determined  based  on  an  expected  value  of  individual  failure.  In  this 
case  it  was  75%.  The  planners  simply  solved  the  equation  X  *  .75  =  6  to  determine  they 
needed  to  start  with  8  helicopters.  The  average  based  decision  method  does  not  take  into 
account  the  "tails"  of  probability  of  success  or  failure.  It  places  you  somewhere  in  the 
middle  of  the  chances  of  succeeding  or  failing. 


organization 
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OA  Distribution: 
pplication  of  Applied  Probability 


*  Assu m  ing  that  the  rel  iabU  i ty  o f  each  hel  icopter  is 
independent  of  each  other  applied  probability  gives  a 
calculation  for  the  overall  mission  success 


To  limit  the 
risk  of  failure  1 
under  1%  o.B 
due  to  hell-  o.fi 
copter  failure  ^ 
12  helicopter  0 
are  necessary 


Probability  of  Successfully  Hiving  b 
or  more  Hclos  Complete  the  mission  - 
vm  u,y?  0-99 


-rar 


?  in  ii  ]i 

mtii.  1  number  of  helos  starting  the  missiftn 
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Instructor  notes: 

Explain  what  a  binomial  distribution  is  by  using  a  simple  example: 

—3  heads  out  of  five  coin  tosses 

—Or  in  this  case  6  or  more  heads  out  of  how  many  coin  tosses  I  need  for  a  certain  level  of 
success 

Discuss  independence  assumption  on  each  trial  and  how  it  this  case  it  is  false  (helos  flying 
in  formation  are  not  independent),  but  this  method  still  gives  one  a  gross  level  assessment 
of  risk  of  failure. 

What  can  be  seen  is  more  resources:  less  risk  of  failure 
Point  out  the  end  of  a  risk  curve  or  S  curve:  more  on  that  later. 
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CS3  OA  Distribution: 

Simulation  Results  (Monte  Carlo) 

p  The  Monte  Carlo  simulation  of  the  helos  as 
independent  events  gives  even  a  little  bit  more 
pessimistic  probabilities  for  success  but  are  in  the 
same  range  as  the  applied  probabilities 


100  Simulations  with  8  Helos:  100  Simulations  with  11  Helos: 

72/1  CHj  Missions  Suocessf ul  94H  00  M  issions  5  uctes  sfu  I 


Slide  5: 

Simulation  is  another  approach  and  can  mitigate  independence  assum  ption.  In  this  case 
we  simulate  the  round  trip  mission  and  record  the  number  of  helos  surviving  after  each 
mission. 

Use  point  to  explain  a  frequency  plot  that  provide  a  histogram 


What  is  Proper  Risk  Assessment? 


+  Risk  Assessment  is  the  identification, 
evaluation,  and  estimation  ol  the  levels  of 
risks  involved  in  a  situation, 

♦  Risk  can  Include  unknown  knowledge  about 
the  situation,  unknown  knowledge  about 
the  development  of  the  course  of  action  or 
unknown  knowledge  about  the  processes 
driving  the  situation. 

*  Historical  data  analysis,  probability  theory 
and  simulation  are  frequently  used  to 
quantify  risk  and  bound  uncertainly. 


■  Fooler  T-kI 
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q  Strengths  and  Weaknesses 
Risk  Assessment 

•  Provides  insight  into  marginal  benefit  of 
resources  to  lessen  risk. 

•  The  better  the  historical  date  base  and/or 
estimates,  the  better  the  assessment  value. 

•  In  many  cases,  assumes  environment  in  the 
immediate  future  will  be  the  same  as  the 
Immediate  past  for  data  consistency. 


■ 


§  Summary 

♦  Risk  analysis  Is  driven  by  you,  the  decision 
maker.  You  define  what  is  an  acceptable 
risk* 

♦  To  best  utilize  your  operational  analyst  to  aid 
in  decision  making,  they  need  access  to 
you,  the  decision  maker  to  know  about  your 
preferences  and  to  be  involved  as  well  in 
the  planning  process  os  during  execution. 

♦  The  operational  analyst  brings  a  variety  of 
tools  (optimization,  simulation,  statistics,  and 
assessment  skills)  to  help  you  with  evidence 
based  decision  making. 

>  *3 
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Quantifying  Risk  by  exploring 


variability 

Let's  look  at  a  simplified  budgeting 
example 


We  want  to  determine  the  risk  of  costs 
over  runs  in  a  gas  turbine  rework  facility 


Consider:  What  are  sources  of  highest 
variability  during  the  budget  execution? 


Slide  10: 

Let's  move  from  the  case  study  to  another  example  of  how  one  can  use  risk  assessmsent. 
Let's  apply  this  concept  to  a  budgeting  forecast  where  we  are  budgeting  for  rework  of  30 
gas  turbine  engines  over  three  months. 

Although  we  will  use  a  rework  example,  the  U.S.  Navy  uses  the  same  process  for 
understand  the  risk  in  budgeting  for  ship  construction. 
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I.M25H)  Arrive 


Gas  Turbine  Rework 
Depot  Example 

ftewurk  . 

V  . 


No 

About 


LM2S00 

Inspected 


f  V.;  Repair?  >  } 

LM2500 

Rework 

Yes 

About 

1 

1/3 


'  Repair 
Tarts 


LM5J500 

LM2S00 

Repair 

Shipped 

*11 


Slide  11: 

Explain  flow  of  work  in  the  rework  depot  and  "cost1'  centers.  Ask  seminar  where  they  think 
the  most  "cost"  variance  is.  Discuss  we  expect  that  to  be  in  the  repair  shop. 
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Quantifying  Risk  by  exploring 
variability 


Possible  costs  (or  outlay)  categories: 

>  Salaries  in  each  workshop 

>  Supplies 

>  Utilities 

>  Contract  Services 

>  Fixed  Costs 


Each  category'  highly  correlated  with  engine 
needing  repair  in  addition  rework* 


Which  might  have  the  greatest 
variability? 


Slide  12: 

Explain  where  cost  can  occur  in  our  example: 
>Salaries  in  each  workshop 
>Sup  plies 

>  Utilities 

>Contract  Services 

>  Fixed  Costs 
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Discuss  where  we  get  the  data  distributions:  historical  analysis  from  past  cost  performance 
(best),  subject  matter  expert  experience  (fair),  no  information  (guess).  In  our  example,  we 
use  historical  data  to  draw  upon  to  simulate  the  process.  We  do  this  to  get  a  'i,'combined',| 
distribution  of  total  costs.  Point  out  this  can  be  done  without  simulation  if  past  data  for 
total  costs  are  available. 

Point  out  this  picture  can  be  of  any  process  that  include  variability  from  each  step.  For 
example,  recall  the  U.S.  Navy  uses  this  technique  for  ship  cost  budgeting.  Each  input 
distribution  are  categories  like  cost  of  copper,  steal,  labor,  schedule  slippage;  etc. 
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Closer  look  at  the  histogram  of  results  simulating  1000  quarters  of  30  engine  repairs. 
Explain  points  on  histogram 


Slide  15: 

The  histogram  allows  us  to  conduct  "what  ifs"  What  if  I  budget  $980 K  for  the  coming 
quarter  to  conduct  30  engine  reworks?  Then.,  based  on  my  historical  data  and  simulation, 
111  have  about  a  5%  chance  of  being  under  budgeted. 
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Risk  "S"  Curve  is  Cumulative  Histogram 


Adding  frequency  from 
Histogram  on  top  of  each 
other 


Slide  16: 

Risk  analysts  like  to  convert  the  frequency  results  into  S  curves  for  ease  of  reading.  Explain 
a  S  curve  is  just  the  cumulative  distribution. 


BB  The  Budget  Risk  "S"  Curve 


t-  li  ]£■"■:  Kd  t  4  \  :  i  \\ 
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With  an  “S"  curve,  I  can  read  my  risk  of  being  on  or  under  budget  directly  from  the  y  axis. 
What  if  we  are  only  given  $900K  to  rework  the  30  engines?  What  are  our  chances  we  will 
be  over  budget?  Ask  the  seminar  what  we  can  do  a  bout  it.  Fewer  engines? 
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Contributions  to  variance  (risk) 

■:  ii  i  i 

Irninir  I'.nlLi  illiki', .Wa 

Repair  Labor  Costs  are  largest 
contributor  to  variance  followed  by 
Standard  Rework  Labor  Costs 
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Simulation  results  can  also  provide  where  your  variability  comes  from.  In  this  chart,  we  see 
the  most  variability  comes  from  the  repair  shop  labor  costs  (as  expected  since  we  don't 
know  in  advance  what  repairs  are  needed).  The  second  most  variability  (almost  16%) 
comes  from  the  standard  rework  shop.  Ask  the  seminar  if  this  makes  sense,  or  do  we  need 
to  look  at  the  reasons. 
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E5E3  Contributions  to  variance  (risk) 


"Tornado  Chart" 
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Slide  19: 

Contributions  to  variance  are  frequently  shown  as  tornado  charts.  They  provide  the  same 
information  in  a  bar  format  and  are  preferred  by  many  decision  makers. 


Risk  Assessment  on  Costs:  A  Cost 


Probability  Distribution 


*  *20 
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The  concept  of  variance  and  risk  can  be  found  in  many  analytical  methods.  This  is  a 
cost  estimation  slide  using  regression  to  estimate  future  program  costs,  and  you 
can  seethe  variance  of  the  estimates  come  from  variance  of  the  input  information. 
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B 

Black  Swans  and  Surprise 

•  What  are  Black  Swans? 

•  Can  we  prepare  for  the  unknown? 

•  Can  we  anticipate  surprise? 

❖Resilience 
❖Flexible 
❖Agile 
❖  Innovative 


Slide  21: 

But,  what  about  preparing  for  events  we  don't  plan  for  or  anticipate?  We  cannot  analyze 
or  simulate  that  which  we  don't  anticipate.  Discuss  concept  of  Black  Swains  and  surprise, 
red  teaming,  and  response.  Allow  seminar  to  bring  up  examples. 


g  Jgflj 

Conclusions 

*  Risk  is  a  component  of 
uncertainty  and  variability 

*  CAREFUL:  Average  Based 
Decision  Making! 
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Conclusions.  Review  major  points  on  uncertainty  and  variability. 
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RecDm mended  readings: 
Against  the  Gods 
Failure  of  Risk  Management 
The  Flaws  of  Averages. 


STO-TR-SAS-098 


ANNEX -233 


ANNEX  TO  SAS-098 


organization 


A.7.2  Decision  Trees 


INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Accounting  for  Uncertainty  in  Operational  Planning  -  Probability  and 

Decision  Trees 

Faculty 

Name(s) 

Ken  McNaught,  Cranfield  University 

Length 

60  -  90  minutes 

Learning 

Objectives 

Understand  the  importance  and  role  of  probability  and  decision  trees  to  aid 
operational  planning. 

Understand  the  difference  between  objective  and  subjective  probability  and 
how  probability  represents  uncertainty. 

Understand  the  difference  between  an  outcome  measurement  and  its  utility. 

Know  how  to  structure  a  decision  tree  to  help  choose  between  alternative 
actions  in  the  face  of  uncertainty. 

Outcomes 

As  a  result  of  this  module,  participants  will  be  able  to: 

Appreciate  the  value  of  probability  and  decision  trees. 

Understand  the  difference  between  objective  and  subjective  probability. 

Content 

Outline 

Example  probability  trees  involving  objective  and  subjective  probabilities, 
e.g.  calculation  of  mission  success. 

Materials 

List  of  Required  Pre-Reading: 

Ideally  Ch  2  on  basic  probability  in  Goodwin  and  Wright’s  Decision  Analysis 
for  Management  Judgment,  but  not  essential. 

Methods 

60%  Lecture  20%  Class  Exercises. 

20  %  Break  Outs. 

Exercise(s) 

Seminar  participants  will  be  asked  to  provide  subjective  estimates  for  various 
probabilities  and  utilities;  use  these  to  construct  and  solve  a  decision  tree, 
and  to  recognize  where  the  limits  of  the  approach  lie. 

Break  Outs 

Yes. 

3-5  participants  in  each  break  out  group. 

Discussion 

Questions 

Validity  of  subjective  probabilities,  bounded  rationality  of  human  decision¬ 
makers,  everyday  examples  of  non-linear  utilities. 
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Probability  and 
Decision  Trees 


Ken  McNaught 
Cranfield  University, 
Defence  Academy  of  the  UK 


•  Draft  for  SAS-098.  RTG-043 


10/28/2013  •  1 


EEffl 

Overview 

•  Situation:  Conditions  preceding  planned  launch  of 
Challenger  Space  Shuttle  were  causing  serious 
concern. 

•  Decision:  Continue  with  Shuttle  launch  or  abort 

•  Objective:  Identify  the  best  decision,  accounting  for 
the  uncertainties  and  value  judgements 

•  OA  Contribution:  Properly  and  objectively  assess  the 
risk  by  carefully  structuring  the  problem 


•  • 


This  provides  an  example  of  the  dangers  of  relying  on  gut  instinct,  rather  than  conducting  a 
quick  but  logical  analysis. 
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Situation 


•  On  28  January,  1 986,  the  Challenger  Space  Shuttle 
was  scheduled  to  launch. 

•  Temperatures  had  reached  a  100-year  low.  The 
morning  of  the  launch  saw  freezing  temperatures  of 
29F. 


®  The  Problem 

•  The  supplier  of  O-rings  for  the  solid  rocket  booster 
(SRB)  joints  feared  that  they  would  not  seal 
effectively  at  low  temperatures  and  the  night 
before,  advised  against  launching. 

•  NASA  requested  them  to  offer  proof  of  the  problem. 
Despite  heated  arguments  within  the  supplier,  the 
supplier's  management  eventually  decided  that 
they  did  not  have  the  proof  requested. 


•  • 
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®  OA  Contribution  1 

•  Data  Analysis 

o  The  engineers  could  have  examined  the 
frequency  of  damage  incidents  over  all  past 
missions  as  a  function  of  temperature  at  launch 
(including  zero  frequencies). 

o  Result:  Based  on  such  a  data  analysis,  there  is  a 
clear  indication  of  greater  damage  at  lower 
temperatures.  However,  omission  of  cases  where 
no  damage  occurred  blurs  the  pattern 
considerably. 

o  Conclusion:  The  data  analysis  needs  to  be 
thoughtful.  Reliance  on  semi-automated  Excel 
graphics  will  not  do. 


Although  temperature  was  known  to  be  a  problem  for  the  O-rings,  it  appears  that  the 
engineers  did  not  consider  plotting  level  of  damage  vs  temperature.  However,  even  if  they 
had,  they  would  have  had  to  include  incidences  of  zero  damage  for  the  graph  to  be  useful. 
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[First  Graph  (only  including 
launches  when  damage  occurred) 


Damage 


Temp(F) 


No  obvious  pattern  given  the  relatively  few  points  plotted. 
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Second  Graph  (all  launches 
included) 


Damage 


The  pattern  of  more  damage  tending  to  occur  at  lower  temperatures  is  much  clearer  in  this 
graph.  By  including  all  of  the  launches,  we  see  that  most  of  the  higher  temperature 
launches  suffer  no  damage. 


®  OA  Contribution  2 

•  Problem  Structuring 

o  Even  without  any  available  data  on  launch 
temperatures  and  damage  incidence, 
subjective  judgements  can  be  usefully  applied. 
This  is  easier  when  the  problem  is  decomposed 
into  its  constituent  parts. 

o  Method:  A  tool  called  a  decision  tree  is  useful  to 
separate  decisions  within  our  control  from 
chance  events  outwith  our  control,  and  then  to 
combine  uncertain  events  with  consequences 

o  Output:  A  structured,  defensible  analysis  based 
on  best  judgments  rather  than  gut  instinct. 

•  •  8 
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These  assessments  are  subjective  but  if  anyone  queries  them,  the  analysis  can  be  done 
with  their  preferred  assessments  instead.  The  result  is  likely  to  be  similar  unless  some  very 
odd  judgments  are  made.  The  assigned  subjective  probability  of  0.3  for  the  occurrence  of 
critical  damage  from  cold  weather  is  deliberately  optimistic  to  try  and  reflect  the  optimistic 
bias  that  existed  pre-launch.  Even  this  optimistic  judgment,  however,  still  results  in  a 
recommendation  not  to  launch.  The  suggested  utility  of  0.99  for  a  mission  being  late  but 
returning  safely  may  also  be  questioned.  The  group  then  has  to  consider  how  many  late  but 
safe  missions  they  would  trade  off  to  avoid  one  disaster.  The  utility  would  have  to  drop  to 
about  0.73  to  reach  the  break-even  point.  Would  anyone  seriously  think  that  the  drop  in 
utility  from  an  on-time  and  safe  mission  was  as  much  as  a  quarter  of  the  drop  in  utility 
from  an  on-time  and  safe  mission  to  the  disastrous  loss  outcome? 


organization 


ANNEX -240 


STO-TR-SAS-098 


NATO 

OTAN 


ANNEX  TO  SAS-098 


i 


I 


SI  Solving  the  Tree 

•  Assume  that  the  payoffs  shown  at  the  right  of  the  tree 
represent  our  values  or  utilities  for  the  various  outcomes. 

•  The  usual  objective  in  such  situations  is  to  maximize  the  so- 
called  expected  value  which  is  actually  just  a  weighted  sum, 
with  the  weights  corresponding  to  the  estimated  probabilities 
of  the  various  outcomes. 

•  The  expected  value  at  each  chance  node  is  calculated,  e.g. 
at  node  2,  it  is  (0.99  x  0.99)  +  (0.01  x  0)  =  0.98  to  2  d.p. 

•  Similarly,  at  nodes  4  and  5,  the  expected  values  are  0.99  and 
0.1,  respectively 

•  Rolling  back  to  chance  node  3,  the  expected  value  is 
(0.7x0.99)  +  (0.3  x  0.1)  =0.72. 

•  Finally,  rolling  back  to  decision  node  1 ,  we  would  choose  the 
branch  which  maximized  the  expected  value,  i.e.  the  Abort 
action  with  an  expected  value  of  0.98. 

•  • 


May  need  to  explain  the  statistical  concept  of  expectation,  which  is  really  just  a  weighted 
average.  Look  out  for  anyone  being  mislead  by  the  literal  meaning  of  'expected'  which  is 
obviously  different  to  the  statistical  meaning. 
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S  OA  Contribution 

Defusing  the  Argument  with 
Sensitivity  Analysis 

o  The  tree  clearly  shows  that  aborting  the  launch  until 
warmer  should  be  the  preferred  decision, 
o  However,  we  can  also  calculate  how  low  the  unknown 
probability  of  sustaining  critical  damage  due  to  the  cold 
has  to  be  before  that  best  decision  changes, 
o  In  this  case,  with  the  value  judgements  assumed,  it  has  to 
be  as  low  as  0.01  or  1%  before  continuing  with  the  launch 
becomes  the  preferred  alternative, 
o  Given  the  degree  of  uncertainty  surrounding  the  O  rings' 
ability  to  seal  at  such  low  temperatures,  highlighted  by  the 
manufacturer’s  initial  recommendation  not  to  launch,  such 
a  low  probability  is  not  credible 


^  OA  Contribution 
Providing  Evidence 

•  Outcome 

NASA  decided  to  go  ahead  with  the  launch. 

73  seconds  into  the  flight,  one  of  the  SRBs 
exploded,  and  the  crew  cabin  detached,  hitting  the 
ocean  three  minutes  later  and  killing  everyone 
onboard. 

CONCLUSION:  Despite  the  various 
uncertainties,  a  structured  analysis  of  the 
situation  was  possible  and  could  have  avoided 
the  disaster. 

•  d3 
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•  Without  thoughtful  structuring  of  a  problem 
as  provided  by  OA,  there  often  seems  no 
alternative  but  to  rely  on  gut  instinct. 

•  To  best  utilize  your  OAs  to  aid  in  decision 
making,  they  need  access  to  you,  the 
decision  maker. 

•  The  OA  brings  a  variety  of  tools  like  optimiza¬ 
tion,  simulation,  statistics  (e.g.  hypothesis 
testing),  and  assessment  skills  to  help  you 
with  evidence  based  decision  making. 

•  4  4 


ESS3 

Learning  Objectives 

•  To  understand  the  difference  between  objective 
and  subjective  probability 

•  To  understand  the  difference  between  an  outcome 
measurement  and  its  value  or  utility 

•  To  appreciate  the  role  which  probability  and 
decision  trees  can  play  in  aiding  operational 
planning 

•  To  know  how  to  structure  a  decision  tree  to  help 
choose  between  alternative  actions  in  the  face  of 
uncertainty 


•  • 


STO-TR-SAS-098 


ANNEX -243 


ANNEX  TO  SAS-098 


organization 


ES33 

Danger  of  Relying  on  Gut  Instinct 

•  High-stakes  situations  demand  cool,  logical 
decision-making. 

•  Reducing  a  decision  problem  to  its  most  important 
constituent  parts  and  assessing  values  for  each  part 
individually: 

o  reduces  the  chance  of  missing  something  important,  and 
o  combats  the  danger  of  being  influenced  by  an 
inappropriate  cognitive  bias  which  can  skew  our  thinking 
without  us  even  being  aware  of  it. 

•  A  small  decision  tree  can  be  assembled  in  minutes 
so  providing  an  instant  decision  is  not  required,  this 
could  be  a  few  minutes  very  well  spent. 


gg  Requirements  of  a 
Decision  Tree 

•  Judgments  concerning  the  probabilities  of 
uncertain  outcomes: 

o  Sometimes  we  will  have  hard  data  to  help  us  estimate 
these 

o  Sometimes  we  will  have  very  little  data  and  will  need  to  rely 
on  a  subjective  judgment 

•  Judgments  regarding  the  relative  worth  of  the 
different  possible  outcomes.  These  value  judgments 
may  be  non-linear  with  respect  to  the  measurement 
variable. 
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Probability 

•  The  best  method  we  have  for  quantifying  and 
dealing  with  uncertainty. 

•  Measured  on  a  scale  from  0  to  1 ,  where  0  means 
logically  impossible  and  1  means  logically  certain. 
Such  extreme  values  should  only  be  applied  when 
the  logical  condition  is  met.  If  an  event  is  possible, 
no  matter  how  unlikely,  it  has  a  non-zero  probability. 

•  The  chance  or  probability  of  some  event  of  interest 
occurring  lies  in  this  interval  between  0  and  1,  with 
the  value  of  the  probability  indicating  how  likely  the 
event  is  to  occur. 

•  • 


H  Probability  Properties 

•  If  event  A  is  more  likely  than  even  B,  then 

o  P (A)  >  P(B) 

•  If  events  A  and  B  are  equally  likely,  then 

o  P (A)  =  P(B) 

•  If  we  can  identify  a  set  of  events  such  that  only  one  of  them 
can  be  true,  then  they  are  said  to  be  mutually  exclusive,  e.g. 
rolling  a  1  or  a  2  on  a  fair  6-sided  die. 

•  For  mutually  exclusive  events, 

o  P(A  or  B)  =  P(A)  +  P(B) 

o  E.g.  Pfrolling  1  or  2)  =  P(  1 )  +  P(2)  =  1  /6  +  1  /6  =  1  /3 

•  When  the  set  of  events  is  both  mutually  exclusive  and 
exhaustive  (covers  every  possible  outcome),  then  the  sum  of 
probabilities  over  all  the  events  adds  up  to  1 . 

o  E.g.  P(  1  or  2  or  3  or  4  or  5  or  6)  =  1 
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IB  Probability  Trees  1 


*  Probability  trees  (also  sometimes  called  event  trees) 
help  us  to  visualise  situations  where  several 
uncertain  events  are  relevant,  particularly  it  a 
sequence  of  events  is  being  considered. 

•  Such  trees  are  commonly  used  in  risk  analysis,  where 
logical  assessments  of  high-stakes  situations  are 
regularly  required. 


Probability  Trees  2 


•  A  probability  tree  provides  an  intuitive  graphical 
representation  of  how  chance  events  which 
depend  on  one  another  are  related. 

•  Nodes  represent  uncertain  propositions  or  chance 
events  while  the  branches  emanating  from  a  node 
represent  the  possible  resolutions  of  that  proposition 
or  the  possible  outcomes  of  that  chance  event. 

•  The  branches  emanating  from  any  node  have 
probabilities  attached  to  them  which  must  sum  to  1, 
i.e.  the  outcomes  represented  by  branches  are 
mutually  exclusive  and  exhaustive. 

•  • 
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•  The  tree  is  conventionally  drawn  from  left  to  right. 

•  The  probabilities  attached  to  the  branches  from  the 
first  node  in  the  tree  are  marginal  probabilities,  not 
conditioned  on  any  preceding  event. 

•  Branches  emanating  from  later  nodes  have 
conditional  probabilities  since  these  outcomes  may 
be  dependent  or  conditioned  on  the  set  of 
outcomes  which  have  preceded  them  in  the  tree. 


The  convention  is  to  order  the  nodes  as  much  as  possible  according  to  the  order  in  which 
information  is  revealed  to  the  decision  maker. 


Example 


•  A  mission  involves  inserting  special  forces 

undetected,  having  them  conduct  a  sabotage 
operation  at  some  facility  and  then  escape 
undetected  :  estimate  the  chance  of  success. 
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Solution  to  Example 


Probabilities  on  the  branches  are  estimates  of  the  success  or  failure 
of  each  component  of  the  mission.  The  probabilities  at  the  leaves  of 
the  tree  show  the  chance  of  each  possible  final  outcome,  so  the 
overall  probability  of  a  completely  successful  mission  in  this  case  is 
0.65  or  65%. 


0.1 


P(Undetected)  x  P(Sabotage  successful  |  Undetected)  x  P(Escape  successful  |  Sabotage 
successful  and  Undetected)  =  0.9  x  0.8  x  0.9  =  0.65  to  2  d.p.  Course  participants  may 
suggest  their  own  values  for  any  envisaged  scenario. 


use 

Making  Choices 

•  Maybe  there  is  a  second  way  to  achieve  the  mission,  e.g.  use 
a  PGM. 

•  So  now  if  we  consider  both  alternatives,  we  have  a  choice  or 
decision  to  make. 

•  Probability  trees  are  readily  extended  to  decision  trees. 

•  Inserting  a  rectangular  decision  node  at  the  start  of  the  tree 
indicates  that  the  commander  has  a  choice  of  which  branch 
to  choose  at  this  point. 

•  Round  chance  nodes  indicate  that  the  outcome  is  not  within 
our  control,  i.e.  there  is  an  unpredictable  element  and  we 
simply  have  to  estimate  how  likely  the  different  outcomes  are 
given  everything  that  has  happened  up  to  that  point  in  the 
tree. 


•  • 
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Decision  Trees 


•  In  a  decision  tree,  however,  final  payoff  values  are 
displayed  on  the  leaves  at  the  right  hand  side  of 
the  tree,  not  probabilities. 

•  Each  possible  path  has  an  associated  payoff 
reflecting  the  value  of  that  outcome. 

•  These  are  payoff  values,  not  probabilities. 

•  In  some  cases,  the  outcomes  will  be  costs  which  we 
wish  to  minimize  rather  than  benefits  which  we  wish 
to  maximize. 

•  Both  objective  and  subjective  probabilities  can  be 
combined  within  the  same  tree. 


EE33 

Objective  Probabilities 

•  When  considering  uncertain  events  such  as  tossing 
coins,  drawing  cards,  etc,  we  employ  objective 
probabilities  which  most  people  can  agree  on. 

•  Since  the  long-run  relative  frequency  of  each 
outcome  provides  an  intuitive  definition  of 
probability  in  such  cases,  there  is  little  room  for 
ambiguity  or  disagreement. 


•  • 
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Subjective  Probabilities  1 

•  Consider  the  following  uncertain  events: 

o  N.  Korea  attempts  to  invade  S.  Korea  within  the  next  3  years 
o  There  is  a  major  terror  attack  within  a  NATO  country,  killing  at  least  100 
people  within  the  next  year 
o  You  break  your  leg  within  the  next  year 

•  Here  we  are  dealing  with  one-off  situations 

•  The  distinction  between  these  and  situations  where 
we  have  objective  probabilities  is  the  lack  of 
repeatability. 

•  Long-run  relative  frequencies  are  usually  not 
applicable  since  the  conditions  surrounding  these 
events  are  unique. 

•  • 
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Subjective  Probabilities  2 

•  However,  if  we  consider  the  events  more  carefully, 
there  may  be  some  more  limited  notion  of 
repeatability  which  can  still  help  to  guide  our 
judgment,  although  there  is  no  ‘correct’  answer. 

•  Discuss  what  these  might  be  for  the  three  events 
suggested  above. 


•  • 


The  notion  is  to  identify  some  appropriate  underlying  baseline  frequency  for  broadly 
similar  events,  where  possible. 

E.g.  for  the  first  one,  we  could  look  at  the  frequency  of  invasion  between  two  neighbouring 
hostile  countries  in  the  last  50  years. 

For  the  second,  we  could  also  look  for  more  recent  historical  examples  of  major  terrorist 
attacks  on  NATO  soil  to  estimate  a  baseline  frequency. 

For  the  third,  we  could  possibly  obtain  figures  from  a  medical  or  insurance  database  and 
then  compare  our  situation  to  the  average,  e.g.  do  we  regularly  take  part  in  dangerous 
sports. 


However,  in  each  case,  the  unique  circumstances  need  to  be  accounted  for  in  the  final 
subjective  judgment.  This  means  adjusting  any  such  baseline  figure  by  an  appropriately 
judged  amount  to  reflect  the  specific  circumstances.  Inevitably,  personal  experience, 
knowledge  and  beliefs  will  all  play  a  part  in  this  process.  Consequently,  different  people  will 
arrive  at  different  subjective  probabilities  which  is  perfectly  reasonable  and  to  be  expected. 
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M  Values  1 

•  Values  and,  in  those  situations  where  attitude  to  risk 
is  explicitly  accounted  for,  utilities,  measure  the 
attractiveness  or  worth  of  a  particular  outcome. 

•  Any  scale  can  be  chosen,  e.g.  0  to  1 , 0  to  1 00  or  -1 
to  1 ,  since  what  matters  is  the  relative  size  of  the 
interval  between  two  values. 

•  If  three  different  outcomes  a,  b  and  c,  respectively, 
are  given  values  of  1 , 0.5  and  0  on  a  0  to  1  scale,  if 
means  that  the  improvement  in  going  from 
outcome  c  to  outcome  b  is  equal  to  the 
improvement  in  going  from  outcome  b  to  outcome 
a. 


May  need  to  emphasise  that  having  an  outcome  with  a  utility  of  1  does  not  mean  it  is  twice 
as  good  as  an  outcome  with  a  utility  of  0.5.  The  scale  is  relative,  not  absolute.  Even  a  utility 
of  0  might  be  OK,  it  is  simply  a  baseline. 


H  Values  2 

•  A  value  of  0.8  for  b  in  this  example  would  mean  that  the 
improvement  in  going  from  c  to  b  is  judged  about  four  times 
as  great  as  the  improvement  in  going  from  b  to  a. 

•  The  high  and  low  endpoints  of  the  scale  typically  correspond 
to  the  best  and  worst  outcomes ,  respectively,  among  the 
possible  outcomes  relating  to  this  decision  situation. 

•  It’s  a  good  idea  to  rank  order  our  preferences  for  all  of  the 
possible  outcomes  identified,  then  attach  values  to  them  as 
described  above. 

•  When  plenty  of  time  is  available,  it  is  recommended  to 
construct  value  functions  but  that  is  not  considered  likely  in  the 
fast  planning  context  we  are  considering. 


Here  we  are  discussing  values  as  though  they  are  one-dimensional,  e.g.  monetary  cost,  but 
in  many  cases  outcomes  will  be  multi-dimensional.  Either  a  vector  of  values  can  be 
presented  for  each  outcome  or,  as  in  multi-criteria  decision  analysis,  we  can  try  to  weight 
the  importance  of  the  different  dimensions  and  collapse  the  vectors  to  aggregate  values. 
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§  COA  Selection 

•  Choice  of  a  Course  of  Action  (COA)  is  a  regular 
problem  commanders  can  expect  to  face. 

•  Can  decision  trees  assist  in  this  task? 

•  The  answer  has  to  be  ‘perhaps’  depending  on  how 
much  time  is  available  to  make  the  decision. 

•  However,  the  more  the  technique  is  practised,  the 
taster  its  use  will  become. 

•  In  this  exercise,  we  require  the  group  to  construct  a 
scenario,  develop  a  decision  tree  to  model  it, 
analyse  the  decision  tree  and  hence  identify  a 
good  COA. 


If  the  group  are  reluctant  or  require  an  example,  an  excellent  case  is  provided  by  Ng  (1993) 
which  can  be  found  on  the  Internet.  But  the  greatest  value  will  be  provided  by  the  group 
taking  time  to  generate  their  own  problem  scenario. 


§  Example  (from  Ng,  1993) 

•  In  this  scenario,  the  enemy  force  initially  consisted  of 
two  divisions  advancing  to  the  West. 

•  However,  the  southernmost  of  these  had  been 
destroyed  by  two  allied  divisions  in  the  South,  now  in 
a  hasty  defensive  posture  to  the  South  of  the 
remaining  enemy  division. 

•  Two  chokepoints  lie  between  the  allied  position  and 
the  enemy  axis  of  advance  thought  most  likely. 

•  It  is  not  known  how  the  enemy  will  react  to  the 
defeat  of  their  southern  division. 

Ng  KW  (1993).  Using  decision  trees  to  direct  the  planning  thought  process:  an 

enhancement  to  the  planning  methodology.  MSc  thesis,  US  Army  Command  & 

©eneral  Staff  College.  • 
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Identify  Both  Sides'  COAs 

•  E.g.  in  Ng ( 1 993),  the  allied  COAs  are: 

o  Remain  in  hasty  defence 

o  Advance  into  chokepoints 

•  Identify  the  allied  and  enemy  COAs  for  the 
scenario  created  by  the  group  at  a  similar 
level  of  detail  to  the  above.  You  will  need  to 
consider  who  is  going  to  act  first  in  the 
chosen  scenario. 


Order  of  COAs 

•  In  some  scenarios,  we  might  be  waiting  for  the 
enemy  to  take  some  action  before  we  respond. 

»  In  such  cases,  our  COA  will  be  dependent  on  theirs 
and  so  will  appear  in  the  tree  after  theirs. 

•  In  most  scenarios,  however,  we  might  expect  to 
take  the  initiative  and  choose  our  COA  before  the 
enemy  has  to  respond  to  it. 

•  Then,  the  enemy  choice  of  COA  will  depend  on 
ours  and  so  appear  in  the  tree  after  ours. 


•  • 
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Describe  the  COAs 

•  For  each  COA  on  each  side,  write  a  brief 
description  of  it  and  what  thinking  might 
lead  the  side  to  take  that  COA. 


Again,  you  could  refer  to  the  example  from  Ng(1993)  if  the  group  needs  help  with  this. 


B 

Building  a  Decision  Tree 

•  Our  side’s  choice  of  COA  is  always  represented  by 
a  decision  node. 

•  The  enemy  choice  of  COA  is  represented  as  a 
chance  node  since  it  is  not  in  our  control. 

•  Probabilities  ot  the  enemy  COAs  are  estimated, 
conditional  on  our  chosen  COA  if  it  comes  after  our 
action,  or  vice  versa  if  before. 

•  Payoff  values  of  the  various  possible  outcomes  are 

also  judged,  typically  although  not  necessarily,  on  a 
0  to  100  scale.  If  people  prefer  to  use  a  scale  from 
some  negative  value  to  some  positive  value,  where 
0  is  neutral,  that  is  also  intuitive.  . 
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Construction  and  Solution  of 
the  Decision  Tree 


•  Draw  the  tree  with  time  passing  from  left  to  right. 

•  Label  each  of  the  branches  and  attach 
probabilities  to  enemy  COAs  (and  any  other 
chance  events  if  any  additional  chance  nodes 
have  been  explicitly  included). 

•  Attach  final  payoff  values  to  each  possible  end 
point  at  the  far  right  of  the  tree. 

•  Use  the  ‘rollback’  method  previously  described  to 
find  the  apparent  best  COA. 


•  Footer  Text 


10/28/2013  •  39 


Sensitivity 


•  Does  the  preferred  COA  from  the  analysis  mainly 
benefit  from  one  particular  component,  resulting 
from  a  particular  enemy  COA? 

•  If  so,  how  much  would  the  probabilities  and  payoffs 
associated  with  that  component  have  to  change 
by  before  this  COA  was  no  longer  preferred? 

•  If  it  is  a  significant  change,  then  the  COA  appears 
robust.  If  not,  the  preferred  COA  may  not  be  so 
clear  cut. 

•  However,  we  must  be  aware  of  the  enemy  taking 
other  measures,  perhaps  not  explicitly  captured  in 
the  tree.  Use  the  tree  to  guide  thinking,  but  don’t  be 
constrained  by  it. 

•  • 
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•  Is  our  second-best  COA  suffering  from  any 
particular  enemy  COA? 

•  If  we  could  close  down  this  enemy  option,  the  other 
options  become  more  likely.  Re-assess  these 
probabilities  and  repeat  the  rollback  to  see  if  our 
preferred  COA  changes. 

•  Of  course,  we  can  try  the  same  thing  with  our 
original  best  COA.  Are  either  of  the  enemy  options 
we  are  thinking  about  influencing  more  easily 
closed? 

•  Consider  these  aspects  when  coming  to  a  final 
choice  of  COA. 

•  • 


For  example,  in  Ng(1993),  the  inferior  'Secure  chokepoints'  COA  suffered  from  the 
possibility  of  the  enemy  diverting  North.  If  that  chance  could  be  reduced,  it  could  become 
the  better  COA.  Is  one  of  the  allied  COAs  which  the  group  have  developed  suffering  from  a 
particular  enemy  COA?  If  so,  can  they  do  something  to  deter  the  enemy  from  taking  that 
COA?  Encourage  the  group  to  consider  what  actions  they  might  be  able  to  take  which  will 
influence  the  enemy  COA  in  ways  which  improve  our  expected  payoff. 
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Implications 

•  Careful  analysis  of  the  tree  can  sometimes  show 
that  closure  of  certain  enemy  options  is  very 
beneficial,  and  resources  could  possibly  then  be 
requested  to  help  with  that. 

•  If  this  is  realistic,  then  the  tree  has  perhaps  helped  to 
suggest  a  more  refined  and  better  COA. 

•  Occasionally,  a  brand  new  COA  might  also  suggest 
itself,  following  the  deeper  thinking  about  the 
problem  situation. 


If  this  doesn't  follow  from  the  group's  tree,  provide  an  example  where  it  does,  such  as 
Ng(1993),  so  that  they  understand  the  point  being  made. 


Conclusion 

•  Problems  involving  risk,  uncertainty  and  value 
judgments  are  inherently  difficult. 

•  Decision  analysis  offers  a  number  of  tools,  including 
the  decision  tree,  to  help  give  some  structure  to 
these  problems. 

•  This  usually  helps  us  to  organise  our  thoughts  and 
think  more  carefully  and  clearly  about  the  choices 
facing  us. 

•  We  will  still  face  difficult  judgments  and  we  should 
not  let  the  tool  ‘make  the  decision’  for  us,  but  we 
should  use  it  to  aid  our  thinking  about  the  problem 
and  to  help  create  new  alternatives. 

•  • 
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Recommended  readings: 

Making  Hard  Decisions,  Clemen  and  Reilly 

Decision  Analysis  for  Management  Judgment,  Goodwin  and  Wright 
The  Failure  of  Risk  Management,  Hubbard 

Using  decision  trees  to  direct  the  planning  thought-process:  an  enhancement  to  the 
planning  methodology,  Maj  KW  Ng,  Masters  thesis,  US  Army  Command  and  General  Staff 
College,  1995.  Accessed  on  24  October  2013  at  URL: 

httDs://www.hsdl.ora/?view&did=460707 
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A.7.3  Expeditionary  Operations  Morphological  Analysis 

INSTRUCTIONAL  MODULE  SUMMARY 


Title 

Analytical  Techniques  for  Creating  Useful  and  Agreeable  Definitions 

Faculty 

Name(s) 

Han  de  Nijs,  ACT,  NATO 

Length 

20  -  30  minutes 

Learning 

Objectives 

Develop  useful  terminology  to  make  a  better  decision. 

Outcomes 

As  the  result  of  this  module,  participants  will  be  able  to: 

Appreciate  the  OA  tools  available  to  define  problems. 

Understand  the  use  of  techniques  to  develop  the  terminology  and  identify 
areas  of  consensus. 

Content 

Outline 

NATO  Expeditionary  Ops  using  the  conventional  approach. 

NATO  Expeditionary  Ops  using  morphological  analysis. 

Classic  and  extended  morphological  analysis. 

Rule  sets  and  analysis  of  results. 

Application  and  OA  contribution. 

Case  Study:  NATO  response  to  Pakistan  earthquake. 

Materials 

List  Required  Pre-Readings: 

None. 

Methods 

80%  Lecture  20%  Class  discussion. 

%  Break  Outs. 

Exercise(s) 

None. 

Break  Outs 

Yes  X  No. 

#  participants  in  each  break  out  group. 

Discussion 

Questions 

In  your  experience,  where  has  hierarchical  clustering  of  costs  and  use  of 
decision  trees  been  used  to  estimate  costs? 
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Analytical 
Techniques  for 
Creating  Useful 
and  Agreeable 
Definitions 

Draft  for  SAS-09B.  RTG-043  7^1 5/2013  1 


™  Overview 


*  Situation:  Problem  formulation 

*  Decision:  Determine  an  agreeable  definition 

*  Objective:  Develop  useful  terminology  to  make  a 
better  decision 

*  Analysis  Cont?i button:  Use  of  techniques  to  develop 
the  terminology  and  identify  areas  of  consensus 
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Problem 


-  Many  organisations  work  via  consensus 

o  NATO  require  it:  26  nations  +  different  subcomponents;  pigs 
industry;  academia 

*  Different  interpretations,  opinions  &  views  on  key 
terms 

o  Pcrlleularty  on  sensitive  Issues 

o  In  NATO:  Counter  Insurgency:  Expeditionary  Operations;  Hybrid 
Threats;  Terroilsm 

*  Consensus  achieved  at  expense  of  utility 

o  5e  nsitive  ■definitions  get  'watered  down1 


This  is  a  problem.,  and  lrm  going  to  show  a  solution  using  the  case  study  'expeditionary 
operations'. 

NATO  -  like  many  organisation  -  works  on  a  principal  of  consensus  -  actually  within  NATO 
consensus  is  much  more  formalised.  Each  NATO  can  effectively  veto  activity 


™  Conventional 
approach 

•  Determine  future  capability  requirements 

o  Create  future  scenarios 

o  Mission-task  analysis... 


5 
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Create  future 
scenarios 


Where  are  we  operating? 
When  are  we  doing  it? 


.  v  , 

f  '  • 

T  4  I 


v  . 


™  Mission-Task  Analysis 

What  are  we  doing? 

How  are  ^ve  going  to  do  it? 
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Conventional 


=  Capability  at  Force  Requirements 
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H  Example  of  conventional 
approach 

NATO  Expeditionary  Operations  are: 
“The  projection  of  military  power  over 
extended  lines  of  communications 
into  a  distant  operational  area  to 
accomplish  a  specific  objechVe.*’ 


AAP  *{2007)  NATOGfaniy  of  Term* and  Definitions  (2003) 


Speaking  notes 

-Current  NATO  definition  -  an  example  of  how  agreement  is  at  the  expense  of  usefulness 
-Traditional  capability  requirements  analysis  -  create  scenarios  -  mission  task  analysis 
-Th  is  definition  covers  all  scenarios 

This  is  the  current  NATO  definition  of  Expeditionary  Operations.  In  conventional  Capability 
Requirements  ana  lysis,  the  normal  process  would  be  to  take  a  concept  or  environment, 
develop  it  into  scenarios,  and  from  those  scenarios  identify  the  capabilities  required 
through  a  mission  /  task  analysis. 

The  problem  is  that  the  definition  is  (for  capability  analysis  purposes)  poor.  The  definition  is 
typical  of  NATO  definitions,  it  is  what  2B  nations  can  agree  on”  it  is  vague  and  it  would 
difficult  to  use  it  to  develop  a  robust  set  of  scenarios. 
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Example  of  conventional 
approach 


NATO  Expeditionary  Operations  are: 

“The  projection  of  military  power  over 
extended  tfnes  of  commim/catfons 
into  a  distant  operational  area  to 
accomplish  a  specific  objective/' 


AAP&C2OT7)  NATO  Glossary  of  Terms  and  Definitions  {2003) 


Speaking  notes 

-Current  NATO  definition  -  an  example  of  how  agreement  is  at  the  expense  of  usefulness 
-Traditional  capability  requirements  analysis  -  create  scenarios  -  mission  task  analysis 
-This  definition  covers  all  scenarios 

This  is  the  current  NATO  definition  of  Expeditionary  Operations.  In  conventional  Capability 
Requirements  analysis,  the  normal  process  would  be  to  take  a  concept  or  environment, 
develop  it  into  scenarios,  and  from  those  scenarios  identify  the  capabilities  required 
through  a  mission  /  task  analysis. 

The  problem  is  that  the  definition  is  (for  capability  analysis  purposes)  poor.  The  definition  is 
typical  of  NATO  definitions,  it  is  what  23  nations  can  agree  on~  it  is  vague  and  it  would 
difficult  to  use  it  to  develop  a  robust  set  of  scenarios. 
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1  Conventional 

1  Approach 

NATO  Definition  1 

Where  are  we 

"...a  distant  operational 

operating? 

area ..." 

When  arc  we  doing  [In  the  future...] 

it? 

What  are  we 

"... accomplish  a  specific 

doing? 

objective." 

How  are  we  going 

.  .project  military  power 

to  do  it? 

over  extended  lines  of 

communications ..." 

11 

Qu 

NATO  Definition 

Specific 

7 

i 

Where? 

"... a  distant  operational 
area ..." 

© 

When? 

[In  the  future...] 

© 

What? 

"... accomplish  a  specific 
objective 

© 

How? 

project  military  power 
over  extended  lines  of 

© 

com munica tions  ..." 
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II  NATO  Expeditionary  Ops 
Vision  &  Framework 

*  Framework  &  vision  included  the  task  for 
NATO  Experts  to  find  a  better  definition  than 
AAR  6  provided 

■  Expeditionary  Operations  are: 

o  Far  from  home:  austere  environments;  rapidly 
deployable,,  .but  also 

o  The  whole  mission-type  spectrum:  Global:  A'  all 
scales;  More  than  Just  deployment  &  entry 
o  Not  a  particular  organisation,  family  of 
equipment  or  tactic... 
o  A  state  of  mind 


Speaking  notes 

-Definition  has  been  developed 
-Lacks  clarity 

-An  operation  may  be  all  missions  across  the  globe  at  all  scales,  but  not  at  the  same  time. 

The  EO  ICT  has  developed  a  'conceptual  vision  and  framework'.  This  document  does 
include  some  useful  indicators,  "...long  lines  of  communication..."  and  "...austere 
environments..."  however  it  also  includes  the  following. 

Whole  mission-type  spectrum  may  include:  Urban  operations,  counter-WIVID,  Information 
Operations,  extended  Force  Protection  etc 

It  may  be  true  that  EO  covers  all  missions,  across  the  globe,  and  at  all  scales,  but  we  believe 
that  any  individual  scenario  is  unlikely  to  combined  the  most  stressing  of  all  these  things  at 
the  same  time. 
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NATO  Definition 

EO  Vision 

Specific? 

Where? 

a  distant 

operational  area../' 

Far; 

austere; 

global 

© 

When? 

\ln  the  future.. .] 

© 

What? 

"... accomplish  a 
specific  objective." 

whole 

mission- 

type 

spectrum 

© 

How? 

"... project  military 
pozver  over  extended 
tines  of 

communications  ../' 

rapidly; 

all  scales; 

© 

Problem 


*  $  c  op  e  of  ex ped  iti  on  ary  o  p  era  t  io  ns  very  b  roa  d 

*  With  current  information: 

o  Diflit  Lilt  to  determine  what  im 1 1  a  NAT O  expeditionary  opera  Non 
o  Impossible  i  o  determine  capability  requirements 
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Venn  Diagram:  which  model? 


All  NATO 
Operations 


1 

k 

rE*f 

O 

L 

edition 

peratioi 

ary 

is  J 

16 
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S  Alternative 
Approach 

♦  Can  we  describe  aH  NATO  future  operations? 

♦  Can  we  distinguish  those  operations  that  we  are 
interested  in? 

♦  Approach 

0  SirVCtUtfE  th#  problem:  MOrphob^iCOl  Analysis 
o  Organize  It;  Classification  Trees 


Speaking  notes 

-focus  on  the  characteristics  that  describe  ED,  but  the  first  step  is  to  identify  all 
characteristics 

-  Morphological  analysis  +  Classification  trees  -  novel  -  no  example  of  this  before 
-Other  techniques  were  considered 

This  is  the  outline  method. 

We  aspired  to  determine  a  sub-set  of  the  NATO  endeavours  that  would  be  considered  to  be 
Expeditionary  Operations.  Clearly,  a  sensible  first  step  was  to  identify  the  full  set  of  NATO 
endeavour. 

Method  selection: 

Why  MA  and  why  classification  trees  -  we  did  look  at  other  techniques  e.g.  Historical 
Analysis  and  traditional  scenario  development 

Combining  the  two  methods  is  considered  novel,  we  have  not  found  any  other  examples  of 
the  employment  of  these  two  techniques  in  such  a  manner. 
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Structuring  a  Problem  with 
Morphological  Analysis 


*  Develop  categories  to  describe  the  problem 

o  Jl  Dimensions" 

*  Identify  different  states  in  each  category 

o  ‘"Stolans11 

*  with  experts  discuss  validity  of  permutations 

□  Morphological  fable 


IS 


Dimensions  &  Scales 


for  All  Operations 

Dimensions 

Scales 

Speed  of  Response 

Ddy^;  weeks:  month* 

Duration 

Enduring;  non-enduring 

piiionee 

Within  NATO:  Out  Of  NATO 

Scale 

Sub-state;  country:  regional;  global 

Mission  Type 

Deiberate;  Intervention:  Focused  intervention; 
Power  Project:  Evacuation:  Humanitarian 
Operation;  Peacekeeping  -  Peace  Enforcement 

Human  environment 

Homogenous:  heterogeneous  integrated: 
henepog^ngou  n  s&gregated 

Physical  environment 

Rural;  urban:  littoral;  ocean;  space;  Arctic 
:  Antarctic 

Infrastructure 

Bare;  austere;  well  found 

Threat 

Permissive;  non -permissive 

Held  a  two  day  workshop  with  representatives  (nations,  operators,  problem  owners, 
analysts)  and  developed  and  agreed  a  set  of  dimensions  and  scalars  that  could  be  used  to 
describe  any  NATO  operation. 
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Morphological  Table 
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Speaking  notes 

-The  SAS  team  came  up  with  this  Morphological  Table 

-lit  has  nine  dimensions  (called  parameters  in  MA}  p  and  a  number  of  scalars  (parameter 
values}  for  each 

-  assumption  that  any  military  operation  can  be  described 

Study  Team  met  (spring  2Q0S)  and  developed  9  dimensions  (parameter  in  MA  speak)  and 
scalars  for  each  dimension  (parameter  values). 

It  was  our  assumption  that  any  military  operation  should  be  able  to  be  described  using  a 
smgle  corn  bi  nation  of  characteristics.  (N  B  At  the  spring  meeting  we  took  nine  historical 
operations  and  attempted  to  describe  them  using  the  dimensions  and  scalars). 


STO-TR-SAS-098 


ANNEX -273 


ANNEX  TO  SAS-098 


organization 


Example:  NATO  Response  to 
Pakistan  Earthquake 


Speer  ot 

R-i  i  iiniu 

Dun  Han 

DI:tjnco 

5ci!t 
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Throat 

DafL 

End. rinn 

W*+ir- 
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&ufc- 

Dcllb trafc 

Ini  c^nHon 

Nomtacntoita 
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Bart 
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Wmtkt. 
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Outfit 
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Font 
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Speaking  notes 

-  MATO  response  to  Pakistan  Earthquake 


§  Classic  Morphological 
Analysis 

*  Manually  inspect  valid  combinations  (<3000) 

*  Workshop  with  6-8 SMEs 

*  Cross  Consistency  Matrix  to  determine  valid 
Combinations 

*  For  example:  Inconsistencies  that  can  be 
eliminated: 

o  Speed:  In  Months  &  Distance;  Withm  NATO 
o  Mission:  Evacuation  &  Duration:  Enduring 
o  Region:  Arctic  &  Scale:  Global 
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Extended 

Morphological  Analysis 

*  Classic  approach  is  not  sufficient 

*  Extended  approach: 

o  Expeditionary  0  ps  combinations  >42 ,000 
o  Large  NATOSME  pool 

q  Dynamic  questionnaire  based  on  Cross  Consistency  Matrix 
leading  to  '"predictor  variables” 
o  Survey  individuals  £,  groups  in  work-shops 
a  Use  of  '“classification  trees’'  to  analyse  results 


Speaking  notes 

-Classic  approach  inadequate 

-Table  too  large,  many  stakeholders 

-Questionnaire  -  can  be  emailed,  or  used  as  a  facilitation  tool 
-Sample  of  data  from  questionnaire  is  limited 

Classic  Approach  =  5  or  6  dimensions,  with  up  to  four  scales,  a  group  of  6  experts  works 
through  all  of  the  combinations  (“4000 )  using  a  Cross  Consistency  Matrix. 

With  a  bigger  problem  (4ZOOO  combinations)  and  larger  number  of  SMEs  to  poll,  we  can't 
do  the  classic  approach,  and  we  extended  it  to  create  a  data  set  that  could  be  analysed. 

Dynamic  Questionnaire  (questionnaire  changes  as  respondent  answers)  reduced  questions 
from  500  to  something  that  can  be  completed  within  20-3 0  mins.  Questionnaire  was  used 
both  individually  and  as  a  facilitation  tool  as  part  of  workshops  with  multiple  SMEs. 

Issued  the  questionnaire... 

Number  of  responses? 

We  modified  our  approach  -  facilitated  sessions..., 
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Analysis  &  Results 


*  Predictor  variables 

*  Classification  Tree  Rules:  Rule  Sets 

*  Definition  accuracy:  precision  and  sensitivity 
Leading  to: 

*  New  definition 


B  Analysis:  Predictor 


Variables 


■  =  Considered  like  to  be  NATO  Expeditionary  Operation 

I  ■  Con  si  dote  d  to  be  N  ATO  E  x  pediti  ona  ry  Opera  Hon 

■  =  Considered  unlikely  to  be  NATO  Expeditionary  Operation 
=  Not  a  discriminating  variable 


Speaking  notes 

-This  is  a  simple  table  showing  the  results 

-Allows  quick  glance  to  see  strong  influences  (Global,  Space,  any  mission  type,  threat  etc) 
-Disadvantage  —  does  not  show  combinations 

It  gives  the  analyst  a  quick  look  at  the  data,,  to  see  strong  influences. 

It  is  also  simple  to  understand. 

It  does  have  disadvantages  though,  the  main  one  being  it  doesn't  show  com  binations 
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■Jffl  Rule  Set 


Qiil  of  N  A- 1  "Or  up  tu  Stjte; 

Ur  bin j  Rural,  Till^ril  Oct-pn;  Bapifi 

of  PfEpDIl-E? 


Speaking  notes 

-By  going  through  a  similar  process,  alternative  definitions  can  be  derived 
-Each  option  represents  a  different  branch  of  the  tree 


Ojrtipn  2=  Li  hnjv- -ii j /LiKli'iv  iriir^  lr net ilt ia 
then  l-rii-ifj/.-r  ipef  d  Jitf  ic^jHinsr 


Rule  Set 


3  =  lf  Willi  in  NATO  them  tin  muni  Uri-in 
Operation  ni  Deliberate  Intervention  CArt  Vl  up  ta 
Ivey  urn 4 1  Seals?  and  Nun- End  cuing,  rapid  spued  of 
ruspufisu 


Oul  uf  N  ATO;  up  to  Regional  Scale;  Urban, 
Rural..  l.i  Moral and  Ocean;  [Rapid  speed  of 


Trsp^nsu 


Oplicm  1  -  Out  i>d'  r-JAlO;  ni|i  iij 
Regional  OmIi-;  Ariilrnr  Antarrlir; 
bare-  ii j  Austere  i n  fra  strut  lure 


Speaking  notes 

-  Opt  3  is  OUT  of  NATO 

-Core  set  +  Opt  i  +  Opt  2  +  Opt  3=  99% 

-Core  set  +  Opt  1  St  2  can  be  combined. 
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Comparing  Definitions 


PRECISION 


Speaking  notes 

*  Core  rule  set  -  better  than  EO  vision  -  less  sensitive  (specific}  but  more  precise 
(coverage)  than  US  MG 

*  Options  -  improve  precision  coverage  at  the  expense  of  sensitivity. 


Example  New  Definition 


"...Operations  conducted  outside  of  the  NATO  nations, 
up  to  Regional  in  scale  and  feature  Cold,  Littoral, 
Ocean,  Rural  or  Urban  environments.  Typically  these 
missions  require  a  response  within  weeks;  a  subset  of 
these  Operations  need  to  be  conducted  in  a  situation 
with  bare  or  austere  infrastructure,  but  the  response 
time  is  reduced  to  months.” 


Proposed  definition  is  longer,  but  has  much  more  detail,  useful  for  capability 
requirements  analysis 
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New  Definition 

Specific? 

Where? 

"...outside  of  NATO  Nations 
up  to  regional  in  scale..." 

© 

When? 

[In  the  future ...I 

© 

What? 

"...in  a  Cold,  Littoral,  Ocean, 
Rural  or  Urban  environment ." 

© 

How? 

"...[quickly]  typically  a 
response  within  weeks ;  or 
Months  with  bare  or  austere 
infrastructure..." 

© 

H  OA  Contribution 


*  Create  more  useful  terminology 

*  Gain  consensus  by: 

q  Highlighting  areas,  of  agreement 
o  Providing  option  where-  fh^r^  i$  disagreemen  \ 

*  Comparison  of  different  definitions 


7/15/2013  SI 
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™  How  did  we  do  it? 

*  Executive  Session  ends  here 

*  What  follows  is  further  detail  on  the  methodology 

*  Only  of  interest  to  the  Analyst  and  Experts 


Structuring  a  Problem  with 
Morphological  Analysis 


*  Develop  categories  to  describe  the  problem 

a  ''D'mensions" 

*  Identifydifferentstatesineachcategory 

o  “Scalars" 

*  Wi  th  exp  erfs  d  i  sc  us  s  val  i  dity  of  perm  uta  f  io  ns 

q  Morphological  table 


33 
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™  Dimensions  &  Scales 
for  All  Operations 


Dimensions 

|  Scales 

Speed  ot  Response 

Days;  weets:  months 

Duration 

Enduring;  non-enduring 

Distance 

Within  NATO ;  out  ot  NATO 

Scale 

Sub-stale;  country;  regional:  global 

Mission  Type 

Deliberate:  Intervention;  Focused  Intervention; 
Power  Project;  Evacuation;  Humanitarian 
Operation;  Peacekeeping:  Peace  Enforcement 

Human  environment 

Homogenous;  heterogeneous  integratea; 
heterogeneous  $agr  egaled 

Physical  environment 

Rural:  urban;  Littoral;  ocean;  space;  Arc  He 
;  Antarctic 

infraslrocture 

Bare;  au$twe:  well  found 

Threat 

Permissive;  non  permissive 

Held  a  two  day  workshop  with  representatives  (nations,  operators,  problem  owners, 
analysts)  and  developed  and  agreed  a  set  of  dimensions  and  scalars  that  could  be  used  to 
describe  any  NATO  operation. 


STO-TR-SAS-098 


ANNEX -281 


ANNEX  TO  SAS-098 


organization 


Morphological  Table 
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Speaking  notes 

-The  S AS  team  came  up  with  this  Morphological  Table 

-It  has  nine  dimensions  (called  parameters  in  MA)  p  and  a  number  of  scalars  (parameter 
values}  for  each 

-  assumption  that  any  military  operation  can  be  described 

Study  Team  met  (spring  2O0&)  and  developed  9  dimensions  (parameter  in  MA  speak)  and 
scalars  for  each  dimension  (parameter  values). 

It  was  our  assumption  that  any  military  operation  should  be  able  to  be  described  using  a 
smgle  combination  of  characteristics.  (NB  At  the  spring  meeting  we  took  nine  historical 
operations  and  attempted  to  describe  them  using  the  dimensions  and  scalars). 
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fill 

Example:  NATO  Response  to 
Pakistan  Earthquake 
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Speaking  notes 

-  NATO  response  to  Pakistan  Earthquake 
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Cross  Consistency 
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Cross  Consistency 
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™  Classic  Morphological 
. . . Analysis., . . 


Speaking  notes 

-Classic  methodology  is  to  agree  on  the  dimensions  of  the  problem 

-  Through  a  series  of  facilitated  workshops 

-  Discuss  combinations  and  come  up  with  a  definition 
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Extended  Morphological 


Analysis 

*  Classic  approach; 

o  Manually  Inspect  valid  combinations  |<3D0G) 

o  workshop  with  6- 8  SMEi 

o  Cross  Consistency  Matrix  to  determine  valid  combinations 

*  Extended  approach: 

q  Expedtlionary  O ps  combinations  > 42.000 
o  Lar^a  NATO  SM.E  pool 

o  Dynamic  quesliannaire  based  on  Cross  Consistency  Matrix 
o  Survey  individuals  fi,  groups  in  work-shops 
o  Use  of  classification  trees  to  analyse  results 


Speaking  notes 

-Classic  approach  inadequate 

-Table  too  large,  many  stakeholders 

-Questionnaire  -  can  be  emailed,  or  used  as  a  facilitation  tool 
-Sample  of  data  from  questionnaire  iis  limited 

Classic  Approach  =  5  or  6  dimensions,  with  up  to  four  scales,  a  group  of  6  experts  works 
through  all  of  the  combinations  {"“4000)  using  a  Cross  Consistency  Matrix. 

With  a  bigger  problem  (42000  combinations)  and  larger  number  of  SMEsto  poll,  we  can't 
do  the  classic  approach,  and  we  extended  it  to  create  a  data  set  that  could  be  analysed. 

Dynamic  Questionnaire  (questionnaire  changes  as  respondent  answers)  reduced  questions 
from  500  to  something  that  can  be  completed  within  20-30  mins.  Questionnaire  was  used 
both  individually  and  as  a  facilitation  tool  as  part  of  workshops  with  multiple  SMEs. 

Issued  the  questionnaire... 

Number  of  responses? 

We  modified  our  approach  -  facilitated  sessions... 
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raSS  Analysis  &  Results 


*  Predictor  variables 

*  Classification  Tree  Rules 

♦  Definition  accuracy 

o  Contusion  matrix 

a  "sensitivity"  &  "precision" 

♦  New  definition 


spDddiar 

DiUHlC* 

llrisT. 
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(He  of 

Wo  C  ■  i 

L-ndurlnq 

NATO 

Hnnlhi  | 

Analysis:  Predictor 
Variable 


Mlbbkrn 


Human 


■  =  Considered  iko  I  to  be  NATO  Expeditionary  Operation 

■  ■  Con  si  d ete  d  to  be  N  ATO  F  x  pediti  ona  ry 

■  OpCMCtiiltered  unlikely  to  be  NATO  Expeditionary  Operation 
=  Not  a  discriminating  variable 


Speaking  notes 

“This  is  a  simple  table  showing  the  results 

-Allows  quick  glance  to  see  strong  influences  (Global.,  Spate;  any  mission  type;  threat  etc) 
-Disadvantage  -doesnotshow  combinations 

It  gives  the  analyst  a  quick  look  at  the  data,  to  see  strong  influences. 

It  is  also  simple  to  understand. 

It  does  have  disadvantages  though,  the  main  one  being  it  doesn't  show  combinations 
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All  Ops 

TH - 75TTT 

Wot  EO  34B26 


Speaking  notes 

-Show  how  classification  tree  builds  {on  LEFT),  Venn  diagram  is  to  Scale 
-irt  level  -  how  many  of  total  combinations  are  considered  Expeditionary 
-Non -expeditionary  may  also  mean  Not- Valid 

-Logical  Contradictions  e.g.  Within  NATO  and  Antartic 
-Empirical  inconsistencies  e.g.  Peacekeeping  at  Global  Scale 
-Normative  Constraints  e.g.  Against  NATO  policy 


Non-expeditionary  operations  may  be  those  that  are  mot  feasible  (i.e.  Antartic,  Within 
NATO)  or  those  that  are  valid  operations  but  not  considered  expeditionary 
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Speaking  notes 
-2nd  Layer 

-Distance  is  most  important 
-Covers  92%  of  ops 


Speaking  notes 
-3rd  Layer 

-Scale  is  ne*t  important 

-Only  4  operations  at  Global  scale  are  considered  valid 


STO-TR-SAS-098 


ANNEX -289 


ANNEX  TO  SAS-098 


organization 


All 
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Speaking  notes 

-Build  up  to  JCore  Rule  Setp 

-Is  good  but  not  good  enough!  It  only  covers  around  58%  of  all  EO 
-Can  use  the  same  approach  to  fill  bit  the  remaining  42% 


■JB  Rule  Set 

/  \ 

Out  at  NATO;  up  tu  Krpunjl  Stdej 

*  Ulhlt,  KwrI,  TillniraE  jnti  Ocwnj  Rapid 
spesd  of  response 


Speaking  notes 

-By  going  through  a  similar  process,  alternative  definitions  can  lb e  derived 
-Each  option  represents  a  different  branch  of  the  tree 
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Out  of  NAT  O:  up  tu  Regjoual  Sale; 
UrbJin,  Rural  l  illorjl  and  Ocpflji;  Kapi.fi 
speed  of  response 


Option  1  -  Out  of  NATO?  up  to 
Re^iuiul  Stale;  And  it  On 


Anln.nrHr;  Einnr  si  r  Auljfchrrr 


infi4isl.mU.un: 


Speaking  notes 
-  Core  set  +  Opt  1  =  79% 


g  Rule  Set 


Out  id  NATO?  up  tu  lie^anriiflL  Sl  jLf; 
Urban,  Rural  littoral  and  Ckean:  Rjptfi 
speed  of  response 


‘v 

Option  1  -  Oul  of  NATO;  up  to  ' 
Re^innisl  Stale;  A  relic  Or 
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re  Spill  nit 


Speaking  notes 

-  Core  set  +  Opt  i  +  Opt  2=  92% 
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Out  of  NATO;  up  tu  Remedial  Scale; 
Uihn,  Rural.  LiHoiul  flnd  Ckwu;  K^pfc?i 
speed  o(  nespoiL-se 


‘v 

Opliou  1  -Dul  of  NATO?  up  to  1 
Re^irmal  Scale;  Aidk  i>i 
Antarctic;  Hnrp  nr  An^herf 
- lillltlS  Ij  ULlWIIL - * 


Oplicai  I -If  bare  or  austere 
infaastmctLite  then  Longer  speed  of 
n?  ip  unit 


Option  3  =  I  f  Within  N  ATO  thuu  a 
Humanitarian  Operation  or  iMWdte 
Intervention  (Art  V)  up  to  Regional  Scale 
and  Nctfi- Enduring  rapid  Speed  Ol  ieipunit 


Speaking  notes 

-  Opt  s  is  OUT  of  NATO 

-Core  set  +  Opt  1  +  Opt  2  +  Opt  3=  99% 

-Core  set  +  Opt!  St  2  can  be  combined. 
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lb  the  atHially? 


(Sumf  E 


♦  Prec/s/on:  A  j  E 

*  Sensitivity:  A  /  F 

•  Model  Accuracy :  (A  +  D)  /  [A  +  B  +  C  +  D} 


Speaking  notes 

if  operation  has  particular  characteristics,  the  generic  definition  will  predict  if  is  EO. 
May  or  may  not  be  correct 

•Want  to  minimise  false  results  and  maximise  correct  predictions 
•Precision  -  the  %  of  Expeditionary  Operations  that  are  covered  by  the  Rule 
•Sensitivity  -  how  many  operations  are  mis-classifiedP  e.g.  If  the  rule  is  that  "All  EO  are 
Out  of  NATO"  that  does  not  mean  that  "All  endeavours  Out  of  NATO  are  EO" 

•Model  Accuracy  maximises  the  correct  predictions  and  minimises  the  false  positives  & 
negatives, 

•Can  measure  existing  definitions 
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Comparing  Definitions 
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Speaking  notes 

Sensitivity  &  Precision  are  formal  terms,  but  easier  to  think  of  Sensitivity  -  ability  to 
avoid  mis- classification  of  EO  as  not  EO  or  vice-versa;  Precision  as  coverage  of  all 
operations  identified  as  NATO  EO. 

*  Current  defin  ition  -  coven  nearly  everyth ing  -  but  does  not  discriminate  very  well 

*  USMC  definition  -  very  sensitive  -  but  misses  key  operations  relevant  to  NATO 

*  OCVStF  -  half  way  between  the  two  (where  we  started) 

*  EO  Vision  -  what  the  Military  Committee  agreed  to  -  least  useful! 
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Comparing  Definitions 


PRECISION 


Speaking  notes 

*  Core  rule  set  -  better  than  EO  vision  -  less  sensitive  (specific}  but  more  precise 
(coverage)  than  USMC, 

*  Options  -  improve  precision  coverage  at  the  expense  of  sensitivity. 
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